Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 24 Jul 2005 00:27:38 +0200
From:      Max Laier <max@love2party.net>
To:        Edwin <edwin@verolan.com>
Cc:        freebsd-hackers@freebsd.org, Giorgos Keramidas <keramida@freebsd.org>
Subject:   Re: help w/panic under heavy load - 5.4
Message-ID:  <200507240027.54127.max@love2party.net>
In-Reply-To: <20050723184108.GA14076@asx01.verolan.com>
References:  <20050719034215.GB20752@asx01.verolan.com> <200507231623.16183.max@love2party.net> <20050723184108.GA14076@asx01.verolan.com>

next in thread | previous in thread | raw e-mail | index | archive | help

[-- Attachment #1 --]
On Saturday 23 July 2005 20:41, Edwin wrote:
> Kernel name: D1-0722 (for reference)
>
> mbsd05#	kgdb kernel.debug /usr/local/STORAGE/crash/vmcore.5
> #13 0xc06933c1 in ip_fastforward (m=0xc12e6c00) at
> /usr/src/sys/netinet/ip_fastfwd.c:572 warning: Source file is more recent
> than executable.

Let's hope that's still correct ...

> 572				if (ip_fragment(ip, &m, mtu, ifp->if_hwassist,
> (kgdb) l
> 567				m->m_pkthdr.csum_flags |= CSUM_IP;
> 568				/*
> 569				 * ip_fragment expects ip_len and ip_off in host byte
> 570				 * order but returns all packets in network byte order
> 571				 */
> 572				if (ip_fragment(ip, &m, mtu, ifp->if_hwassist,
> 573						(~ifp->if_hwassist & CSUM_DELAY_IP))) {
> 574					goto drop;
> 575				}
> 576				KASSERT(m != NULL, ("null mbuf and no error"));
> (kgdb) i loc
> ip = (struct ip *) 0xc12f700e
> m0 = (struct mbuf *) 0xc12f700e
> ro = {ro_rt = 0xc11f8420, ro_dst = {sa_len = 16 '\020', sa_family = 2
> '\002', sa_data = "\000\000ĀĻ\002\005\000\000\000\000\000\000\000"}}
> dst = (struct sockaddr_in *) 0xc76bfc3c
> ia = (struct in_ifaddr *) 0x0
> ifa = (struct ifaddr *) 0x0
> ifp = (struct ifnet *) 0xc0f91800
> odest = {s_addr = 84060352}
> dest = {s_addr = 84060352}
> sum = 0
> ip_len = 0

This should not happen. ip_len is initialize from ntohs(ip->ip_len) and never 
touched again.  Anyway, let's look some more ...

> error = 84060352
> hlen = -1057417216
> mtu = 0
> __func__ = "ip_fastforward"
> (kgdb) p *ip
> $1 = {ip_hl = 5, ip_v = 4, ip_tos = 0 '\0', ip_len = 10240, ip_id = 61249,

ip_len should be 40 as ip_len is supposed to be in HOST BYTE ORDER at this 
point.  Feeding 10240 to ntohs() give the correct value, so something 
obviously went wrong.

Let's see how we got here:
355 does the byteorder flip to host byte order
366 pfil OUT
451 pfil IN
527 first check ip_len < if_mtu etc ...

Obviously, the only thing that might mess with the byte order (unless I missed 
something along the way) is one of the pfil consumers.

***
*** What firewall(s) are you running with?
***

> ip_off = 0, ip_ttl = 63 '?', ip_p = 17 '\021', ip_sum = 31921, ip_src =
> {s_addr = 67479744}, ip_dst = {s_addr = 84060352}} (kgdb) p *m
> $2 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, mh_data = 0xc12f700e "E",
> mh_len = 40, mh_flags = 3, mh_type = 1}, M_dat = {MH = {MH_pkthdr = {rcvif
> = 0xc0f90000, len = 40, header = 0x0, csum_flags = 769, csum_data = 0, tags

40, there you have it - no need to fragment at all!

> /usr/src/sys/netinet/ip_output.c:967
> 967			m->m_next = m_copy(m0, off, len);
> (kgdb) l
> 962				len = ip->ip_len - off;
> 963				m->m_flags |= M_LASTFRAG;
> 964			} else
> 965				mhip->ip_off |= IP_MF;
> 966			mhip->ip_len = htons((u_short)(len + mhlen));
> 967			m->m_next = m_copy(m0, off, len);
> 968			if (m->m_next == NULL) {	/* copy failed */
> 969				m_free(m);
> 970				error = ENOBUFS;	/* ??? */
> 971				ipstat.ips_odropped++;

Just to make sure, we didn't touch the original packet at this point so the 
above values are still the ones we based the (wrong) decision on.

-- 
/"\  Best regards,                      | mlaier@freebsd.org
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier@EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News

[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (FreeBSD)

iD8DBQBC4sRqXyyEoT62BG0RAh+KAJ94YzIMUpdJ4uevZzhCaKBTwp+zswCfblzk
issKUmM+7rkv7Ir28mgPI5E=
=HYJP
-----END PGP SIGNATURE-----

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200507240027.54127.max>