From owner-freebsd-net@FreeBSD.ORG Mon Dec 10 23:18:46 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD229523; Mon, 10 Dec 2012 23:18:46 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6D9028FC12; Mon, 10 Dec 2012 23:18:46 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id wz12so2289828pbc.13 for ; Mon, 10 Dec 2012 15:18:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=U1bh6ZSvMw9khWC+8oQT5O6hNcR+B9frvdyIdwe0I6Q=; b=qkBLsSI2KaCuHhxua9JfeJv/fRq9upt6KNGewi2M5Wk35BxdOhjsFzHuOP0hCGp1Lh BuhDPqizYWnjN3Asmy8GK3rLa2SxjV/UMhSWSdriEmroIfgNRaT6UziJr/STCwSreTqC m4QQBb3CScKR14CSHLmYlVzhbpGzThGKs22jvVjJaqUpFKVhPCDcvntylM6nT6Cp4mpH B5JwYv5LuwRZinziDTgvJffTndClxTp/p4PsB7fAoeNiCTtT1g5/fVsK2B2vuUTiN8dt xXcFM8niQwhGiKizujdSWK1xB826aO+31rEqqb9/YgdKXvTCD9t2BJnoD1tQfWhImR75 9gVg== MIME-Version: 1.0 Received: by 10.66.80.68 with SMTP id p4mr39868443pax.35.1355181525767; Mon, 10 Dec 2012 15:18:45 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.55.166 with HTTP; Mon, 10 Dec 2012 15:18:45 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 15:18:45 -0800 X-Google-Sender-Auth: j0Wm823QFKo7Tm0elWmqQK7-aR0 Message-ID: Subject: Re: "Memory modified after free" - by whom? From: mdf@FreeBSD.org To: Adrian Chadd , Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, FreeBSD Current X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 23:18:46 -0000 On Mon, Dec 10, 2012 at 3:10 PM, Adrian Chadd wrote: > 9216 sounds like a jumbo frame mbuf. So the NIC is writing to an mbuf > after it's finalised/freed. > > I have a similar bug showing up on ath(4) RX. :( Compile with DEBUG_MEMGUARD in the kernel configuration, and then set vm.memguard.desc to the name of the UMA zone used for the 9216 byte allocations, mbuf_jumbo_9k. This should cause a panic when the memory is touched after free. Cheers, matthew > On 10 December 2012 14:22, Garrett Cooper wrote: >> I noticed this while checking the logs on one of my test boxes >> after restarting the network. Any idea where I should start looking >> into this (has IPv6 enabled but wasn't using it, em/cxgbe/ixgbe >> interfaces with the ixgbe interfaces lagged previously, but now not)? >> It looks suspiciously like the same size as a jumbo frame, but I'm not >> 100% sure if that's the real problem. >> Thanks, >> -Garrett >> >> Dec 10 14:03:12 wf158 kernel: em0: link state changed to DOWN >> Dec 10 14:03:13 wf158 kernel: ix0: link state changed to DOWN >> Dec 10 14:03:13 wf158 kernel: ix0: link state changed to UP >> Dec 10 14:03:13 wf158 kernel: ix1: link state changed to DOWN >> Dec 10 14:03:13 wf158 kernel: ix1: link state changed to UP >> Dec 10 14:03:13 wf158 kernel: ix0: link state changed to DOWN >> Dec 10 14:03:13 wf158 kernel: ix0: link state changed to UP >> Dec 10 14:03:15 wf158 kernel: em0: link state changed to UP >> Dec 10 14:03:15 wf158 dhclient: New IP Address (em0): 10.7.169.89 >> Dec 10 14:03:15 wf158 dhclient: New Subnet Mask (em0): 255.255.240.0 >> Dec 10 14:03:15 wf158 dhclient: New Broadcast Address (em0): 10.7.175.255 >> Dec 10 14:03:15 wf158 dhclient: New Routers (em0): 10.7.160.1 >> Dec 10 14:03:16 wf158 kernel: ix0: link state changed to DOWN >> Dec 10 14:03:16 wf158 kernel: ix0: link state changed to UP >> Dec 10 14:03:31 wf158 kernel: in6_purgeaddr: err=65, destination >> address delete failed >> Dec 10 14:03:31 wf158 dhclient[4539]: My address (10.7.169.89) was >> deleted, dhclient exiting >> Dec 10 14:03:32 wf158 dhclient[4521]: short write: wanted 20 got 0 bytes >> Dec 10 14:03:32 wf158 dhclient[4521]: exiting. >> Dec 10 14:03:33 wf158 kernel: em0: link state changed to DOWN >> Dec 10 14:03:33 wf158 kernel: ix1: link state changed to DOWN >> Dec 10 14:03:34 wf158 kernel: ix1: link state changed to UP >> Dec 10 14:03:34 wf158 kernel: ix1: link state changed to DOWN >> Dec 10 14:03:34 wf158 kernel: ix1: link state changed to UP >> Dec 10 14:03:34 wf158 kernel: ix0: link state changed to DOWN >> Dec 10 14:03:34 wf158 kernel: ix0: link state changed to UP >> Dec 10 14:03:34 wf158 kernel: ix1: link state changed to DOWN >> Dec 10 14:03:34 wf158 kernel: ix1: link state changed to UP >> Dec 10 14:03:36 wf158 kernel: em0: link state changed to UP >> Dec 10 14:03:36 wf158 dhclient: New IP Address (em0): 10.7.169.89 >> Dec 10 14:03:36 wf158 dhclient: New Subnet Mask (em0): 255.255.240.0 >> Dec 10 14:03:36 wf158 dhclient: New Broadcast Address (em0): 10.7.175.255 >> Dec 10 14:03:36 wf158 dhclient: New Routers (em0): 10.7.160.1 >> Dec 10 14:05:34 wf158 kernel: Memory modified after free >> 0xffffff81c016d000(9216) val=ffffffff @ 0xffffff81c016d000 >> Dec 10 14:05:35 wf158 kernel: Memory modified after free >> 0xffffff81b5cdc000(9216) val=ffffffff @ 0xffffff81b5cdc000 >> >> # uname -a >> FreeBSD wf158.west.isilon.com 10.0-CURRENT FreeBSD 10.0-CURRENT #1 >> r+2760369-dirty: Mon Dec 10 08:04:46 PST 2012 >> root@wf158.west.isilon.com:/usr/obj/usr/src/sys/ISI-GENERIC amd64 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"