From owner-freebsd-net@freebsd.org Tue Jun 27 05:13:22 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FA43DA0F91; Tue, 27 Jun 2017 05:13:22 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 911E87BDAB; Tue, 27 Jun 2017 05:13:21 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id b184so15089553wme.1; Mon, 26 Jun 2017 22:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mXHIc7XZRwI1tVFy4U5Y05uxFRGexqrwlKrlqodjHzQ=; b=gt06FYWtXnbbSbtYM0mL22+dmjpcOIyjXhwYTMigK7VZyiZw95g0tsF6c3jj2E4otS eBXiXymwMSSF1NfP7zNoGNmCUUKwqBFXbf05MBlVK278wQfa2qpoA1Y95HBTNeqXs27b yGMnoi7c3jtirbCQ7Yqv14OHMPwcaZSuamfLnlX7qwXqIMdNYhwTAjqY209Ud40T7atr oGEIJEeKzy/HzYOyJuE0qhqeBN/gxxffUAInsPq73cdSkjuY9ztTPm29d2FK+VSlkCHf 3/ASf7yuP6sygy5pqVr6S4Vjds/LQi6kSXYmmBTJOKaVywwmFWK8lNkXpz8NX/Er5ruv HYpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mXHIc7XZRwI1tVFy4U5Y05uxFRGexqrwlKrlqodjHzQ=; b=grHq/+mrV2SgKsu7J9xQy9WLaWGyTONi+ZMRSJOe8cCGzH9FFuIDHrxH4W2IksaHK6 hTi3XUFv4m4ts0BZUlKSshK2UaEjijvbxr8F+hGaifC9k10h6qyomFaMyzJ02S1zVDqs op9VK6ZMSjsQP4Iehg5LaHsVCBhRUJCrDEV0pEgORbG70R4GZTuEgp+pSSnFuf+wB9bW P2IzsbZcD+9QDxWlzyKOvxvG+mQWBe2dBrlFS8cOkrDDyCaPwCPBHIXrsQlpgnwZIPVC PBmF30gCT3nUMAoXNUHX19a1H9lLAFSreDMgpV/y5U7D1Fsh0dPiiMphjZxi5mgkiM3R JLjw== X-Gm-Message-State: AKS2vOxIFQ1anzdM7Bxn/6U7/R4PZc+VQLe5qnDNFHHDpRGvt/YgCnS7 rR1M2Q/t+1PWplq/l7ADtwbJiVK42A== X-Received: by 10.80.161.69 with SMTP id 63mr2469430edj.142.1498540399677; Mon, 26 Jun 2017 22:13:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.183.176 with HTTP; Mon, 26 Jun 2017 22:13:18 -0700 (PDT) In-Reply-To: <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> From: Zaphod Beeblebrox Date: Tue, 27 Jun 2017 01:13:18 -0400 Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 05:13:22 -0000 Don't forget that, generally, as I understand it, the network stack suffers from the same problem for 9k buffers. On Sun, Jun 25, 2017 at 12:56 PM, Ben RUBSON wrote: > > On 25 Jun 2017, at 17:32, Ryan Stone wrote: > > > > Having looking at the original email more closely, I see that you showed > an mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > clusters increase while you are far below the zone's limit means that > you're definitely running into the bug I'm describing, and this bug could > plausibly cause the iSCSI errors that you describe. > > > > The issue is that the newer version of the driver tries to allocate a > single buffer to accommodate an MTU-sized packet. Over time, however, > memory will become fragmented and eventually it can become impossible to > allocate a 9k physically contiguous buffer. When this happens the driver > is unable to allocate buffers to receive packets and is forced to drop > them. Presumably, if iSCSI suffers too many packet drops it will terminate > the connection. The older version of the driver limited itself to > page-sized buffers, so it was immune to issues with memory fragmentation. > > Thank you for your explanation Ryan. > You say "over time", and you're right, I have to wait several days (here > 88) before the problem occurs. > Strange however that in 2500MB free memory system is unable to find 9k > physically contiguous. But we never know :) > > Let's then wait for your patch ! > (and reboot for now) > > Many thx ! > > Ben > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >