From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 21 14:47:14 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AEC3D16A421 for ; Thu, 21 Jul 2005 14:47:14 +0000 (GMT) (envelope-from keramida@freebsd.org) Received: from aiolos.otenet.gr (aiolos.otenet.gr [195.170.0.93]) by mx1.FreeBSD.org (Postfix) with ESMTP id 25F5A43DA5 for ; Thu, 21 Jul 2005 14:46:47 +0000 (GMT) (envelope-from keramida@freebsd.org) Received: from beatrix.daedalusnetworks.priv (aris.bedc.ondsl.gr [62.103.39.226]) by aiolos.otenet.gr (8.13.4/8.13.4/Debian-1) with SMTP id j6LEkirZ004219; Thu, 21 Jul 2005 17:46:45 +0300 Received: from beatrix.daedalusnetworks.priv (localhost [127.0.0.1]) by beatrix.daedalusnetworks.priv (8.13.3+Sun/8.13.3) with ESMTP id j6LEki4x019391; Thu, 21 Jul 2005 17:46:44 +0300 (EEST) Received: (from keramida@localhost) by beatrix.daedalusnetworks.priv (8.13.3+Sun/8.13.3/Submit) id j6LEkiWS019390; Thu, 21 Jul 2005 17:46:44 +0300 (EEST) Date: Thu, 21 Jul 2005 17:46:44 +0300 From: Giorgos Keramidas To: Edwin Message-ID: <20050721144644.GA19340@beatrix.daedalusnetworks.priv> References: <20050719034215.GB20752@asx01.verolan.com> <200507191120.37526.jhb@FreeBSD.org> <20050720020302.GA24474@asx01.verolan.com> <20050720100623.GA1470@beatrix.daedalusnetworks.priv> <20050720154156.GA26755@asx01.verolan.com> <20050721115719.GK16179@beatrix.daedalusnetworks.priv> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050721115719.GK16179@beatrix.daedalusnetworks.priv> Cc: freebsd-hackers@freebsd.org Subject: Re: help w/panic under heavy load - 5.4 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jul 2005 14:47:14 -0000 On 2005-07-21 14:57, Giorgos Keramidas wrote: > On 2005-07-20 11:41, Edwin wrote: > > I'm trying to understand the particulars about this - I get the null pointer > > part, but as to ip_fragment - it's fragmenting mbufs to handle ip packets > > during switching? and its failing trying to copy data past the end of the > > chain? > > ip_fastfwd() thinks that it should fragment the packet because it somehow > calculates a bogus ``mtu'' value. See the mtu value in frame 12 of the stack > trace below. > > > #10 0xc0611fef in panic (fmt=0xc0820008 "default") at /usr/src/sys/kern/kern_shutdown.c:550 > > #11 0xc0641a2c in m_copym (m=0x0, off0=1500, len=1480, wait=1) > > at /usr/src/sys/kern/uipc_mbuf.c:385 > > #12 0xc069b694 in ip_fragment (ip=0xc11bd80e, m_frag=0xc76bfc6c, mtu=-1056787456, > > if_hwassist_flags=0, sw_csum=1) at /usr/src/sys/netinet/ip_output.c:967 > > The ``mtu'' is an extremely small integer value, which is definitely a problem > here. Somehow, ip_fastforward() calculates a very wrong value for the ``mtu''. The check for finding the right MTU in ip_output.c is a bit different, as it includes a check for RTF_UP: 777: if (ip->ip_off & IP_DF) { 778: error = EMSGSIZE; 779: /* 780: * This case can happen if the user changed the MTU 781: * of an interface after enabling IP on it. Because 782: * most netifs don't keep track of routes pointing to 783: * them, there is no way for one to update all its 784: * routes when the MTU is changed. 785: */ 786: if ((ro->ro_rt->rt_flags & (RTF_UP | RTF_HOST)) && 787: (ro->ro_rt->rt_rmx.rmx_mtu > ifp->if_mtu)) { 788: ro->ro_rt->rt_rmx.rmx_mtu = ifp->if_mtu; 789: } 790: ipstat.ips_cantfrag++; 791: goto bad; 792: } The check for RTF_UP doesn't exist in ip_fastfwd.c, except perhaps through the ip_findroute() call. I'm probably confused, but it seems that ip_findroute() in ip_fastfwd.c may still return a route entry for a gateway that is not yet RTF_UP, since the check for the RTF_UP flag is not done for the dst.rt_gateway route entry too. This may be the cause of the invalid MTU value you're seeing. Can you try the following patch for ip_fastfwd.c? The diff is also available online at: http://people.freebsd.org/~keramida/diff/fastfwd-mtu.patch %%% Index: ip_fastfwd.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_fastfwd.c,v retrieving revision 1.28 diff -u -r1.28 ip_fastfwd.c --- ip_fastfwd.c 4 May 2005 13:09:19 -0000 1.28 +++ ip_fastfwd.c 21 Jul 2005 14:38:35 -0000 @@ -537,12 +537,13 @@ } /* - * Check if packet fits MTU or if hardware will fragement for us + * Check if packet fits MTU or if hardware will fragment for us. + * If necessary, update the MTU of the route entry too. */ - if (ro.ro_rt->rt_rmx.rmx_mtu) - mtu = min(ro.ro_rt->rt_rmx.rmx_mtu, ifp->if_mtu); - else - mtu = ifp->if_mtu; + if ((ro.ro_rt.rt_flags & (RTF_UP | RTF_HOST)) && + ro.ro_rt->rt_rmx.rmx_mtu > ipf->if_mtu) + ro.ro_rt->rt_rmx.rmx_mtu = ipf->if_mtu; + mtu = ifp->if_mtu; if (ip->ip_len <= mtu || (ifp->if_hwassist & CSUM_FRAGMENT && (ip->ip_off & IP_DF) == 0)) { %%%