From nobody Fri Dec 3 21:17:49 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9F3D118BE87C for ; Fri, 3 Dec 2021 21:18:01 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J5Qf13y6Hz3D70 for ; Fri, 3 Dec 2021 21:18:01 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pf1-x435.google.com with SMTP id z6so4013273pfe.7 for ; Fri, 03 Dec 2021 13:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=25ZirwTo6VLsA32JAuLfA+Rz/CisDSBaTQtATZ5zOFY=; b=baBDM7NB69RqE2GR/Ta3/nWnaA9CG6mMxfv89TLFWBmn4AZULdELgpT2dsdM4MzWNp 0lL5JT955MZKerxknqwleNRP3k9YsQ3fbBJoKQLBTPhgPfw3Onqum8s39UStXfmjg6Pv r21D4oJ33JhwYKQni46sSjwCNS29rnd2rYjk0ZcunewK8s+/LoitqbMg+trDhMEI+rAs WIQoCXKewRug723b1I+GBJkTUpcuVNo7rTM3CqZhP2LGDgg6vM/nTrPsH5RJM3tpZv/K tBXxWFO38X+RHFfr2OOEPyHb7PtWaAf+0AKBUnH0YHecHqRBGl7yH84BsyDcV7A5/l7+ N4oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=25ZirwTo6VLsA32JAuLfA+Rz/CisDSBaTQtATZ5zOFY=; b=bcG2L8Jn2BTTUTc7vJVQq7WiARdC+kgfgLuOpACeodBM0G8XdYDlfGPj23qLt1R+w1 fQWx9Vor36KW+o8eVsfJ2q636Ft08WFiGSQo3DMp3aKOnFe+QItejRHZGwXHZCvNRDdz zJ98iML+180flcHg2TaZHkourhAO8oL/NrTJA6bVWwRbXIbkrWAaMazeBnPhGSqMtr+E cRdU9xcno5KimlJqPp/0KCsSQ6yjs82T9omlqKuoZiqHv0pStmDiSj9MuEzTzAatwnbM CiYsLATXADtD7x07jQl0hvVDuTHzIOzz0vmPuqtkk3eShy4yn9albO4Bd+o8lWInc12g 4v+w== X-Gm-Message-State: AOAM531zJV4p5dy7NaiWMPpEDPqs2osaNFvQ6mv1WXWOxIamGfRUPbLp E4JsT2T/OA+N+xzyhDnLHEi9nUVcWcm9Ci4ZU9DNKLnV X-Google-Smtp-Source: ABdhPJzPLeGLtYd11t9SbX0neuDGFTh3WGABxF9WRFsZ2t0d6bMbtr+KdYrOwnUXNfxbjB3TOqO/Wm01WXS29rfleEU= X-Received: by 2002:a63:3446:: with SMTP id b67mr128013pga.424.1638566280595; Fri, 03 Dec 2021 13:18:00 -0800 (PST) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ryan Stone Date: Fri, 3 Dec 2021 16:17:49 -0500 Message-ID: Subject: Re: Chelsio cards, jumbo frames, memory fragmentation and performance in FreeBSD 13.x? To: John Jasen Cc: FreeBSD Net , Navdeep Parhar Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J5Qf13y6Hz3D70 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Set it to 4k and move on. I'm not aware of any efforts in the VM layer or network stack to make >PAGE_SIZE clusters usable in practical situations. Honestly, at this point I wish that we'd just kill them entirely. On Fri, Dec 3, 2021 at 12:52 PM John Jasen wrote: > > Pretty close to two years ago, we tripped across conditions where heavily > used FreeBSD 11.x packet-filter firewalls would slow to a crawl and load > would go crazy. > > In a fit of hopefulness with our upgrade to FreeBSD 13.0, I removed the > hw.cxgbe.largest_rx_cluster settings we put in place -- only to have our > hardest hit firewall experience high load and plummeting bandwidth within a > few hours. > > So, I have two questions -- is there a better fix for this? Or, is the > approach to set largest_rx_cluster to 4k and move on? > > Thanks in advance!