From nobody Sun Feb 25 23:51:10 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TjgWH2GyXz5Bc2b for ; Sun, 25 Feb 2024 23:51:23 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TjgWH1k6Lz4mMN; Sun, 25 Feb 2024 23:51:23 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-299b92948a6so1569972a91.3; Sun, 25 Feb 2024 15:51:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708905082; x=1709509882; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/pFaXe/N4QPFSwlgMSmhUcoDksJwwzznjZ5QtKvXoeo=; b=SwIb/9yDU2V94bFAf3znVa2S6Wt9NBSBtCaUW1ajGReAjrnpdzcEacFBGTtBeB33mR 0XLOyt6kpIAohb3Udv23aMp36vhzfvqvZwmAscyx+IR3UMHLMBGcgagwYACw9AI65FYR NhASoLx/eMYjRQvjupzfOAOHKgOgGve0p74PSuA3v3QAh79dfEC+O8jY8mU3Kn+w5F6T wbbuoF5x6T6dpIqgwhcyDKD4nCHHxyLNatmb61VILrNU/a7VBcA9xMMMZoLr4iOqr/Wy cZRvvyUF1e1vbfNQz8kiLpS+Q7t3Wdh3x6wRwM31LlqxGLN7iOsTifFe5NIobPxcYF0c gKVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708905082; x=1709509882; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/pFaXe/N4QPFSwlgMSmhUcoDksJwwzznjZ5QtKvXoeo=; b=g9QQ3VtW/TaJ3igYhor1bQR7uFTcI02UKjHPPUty6a0TGW4Uexry8cNC7ltQAk9HZg J2Vf+SV4tKrvXtrAjTn25zr6ZniGk+lyrO378/fGaY1QN349VW1OlLDFLwxilegU7obt FGnorll6w7DdCOZrQAQADVnse6jhQFRnyfBaUS03MkEPcqWMr7SyA5dpAvMsqJQ1qp+c PCIcpeVNoPqHTrLfCwWNYazChJrNg9RkUZv8LxIfnIAF5Qs25KnOj8I1dphqyDua7/4X ddni0gfnnf1LOtx5s7RCt9KcIFVc1jJrlBUPxcSf8wa51G+S4nMe4OJJiodHafQitfNz w5fg== X-Forwarded-Encrypted: i=1; AJvYcCU0du2uUHQY+sA3zbj1yPdeYssgfqN30kM8xhntk60cDzbgsokqKYxNGQL6IKPgpCmWG/mOFqRTO9B1wKRxQGff3avZP1XImA== X-Gm-Message-State: AOJu0YwbXbkvYBFo0Ht8/9WBmYMULej8uqYHmAyRlo536e2/FbTJQvcS OQN7BuNv0kpj6NDHrf/g1nRGhVgwtISYtQ1S3nqgAiyFtkWxI3CXYZ3XLPloQcIsNXkDcJR9W3E B2Ecr1El60Cuzr7+/SCV/Z8A30qTUQ7U= X-Google-Smtp-Source: AGHT+IHzd9jiuuw2dL27Qvpl1i0irCf0K6duaxBheji7fdNjkSB9xFODbkRwn/kDixuP4c2vzuIKKSarh+LwXcGPqg4= X-Received: by 2002:a17:90a:6b0f:b0:29a:b342:6cb5 with SMTP id v15-20020a17090a6b0f00b0029ab3426cb5mr973581pjj.37.1708905082059; Sun, 25 Feb 2024 15:51:22 -0800 (PST) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: <034cc6ea-26d8-4520-879a-672459832407@fsfe.org> <9066A50F-26DC-4314-B79E-66120A2B5A2F@freebsd.org> In-Reply-To: <9066A50F-26DC-4314-B79E-66120A2B5A2F@freebsd.org> From: Rick Macklem Date: Sun, 25 Feb 2024 15:51:10 -0800 Message-ID: Subject: Re: NFS performance with 10GBase-T To: tuexen@freebsd.org Cc: Hannes Hauswedell , "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; TAGGED_RCPT(0.00)[lists2024]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4TjgWH1k6Lz4mMN On Sun, Feb 25, 2024 at 1:21=E2=80=AFAM wrote: > > CAUTION: This email originated from outside of the University of Guelph. = Do not click links or open attachments unless you recognize the sender and = know the content is safe. If in doubt, forward suspicious emails to IThelp@= uoguelph.ca. > > > > On Feb 25, 2024, at 01:18, Hannes Hauswedell wr= ote: > > > > Hi everyone, > > > > I am coming here from > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2771971160 > I guess this should read: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277197 Btw, what Hannes reported in the PR was that performance was about the same for Linux and FreeBSD NFS clients when the link was using a 1500byte ethernet frames. However, Linux performs much better with 9K jumbo frames whereas FreeBSD performance does not improve for 9K jumbo frames. (Some mount options I suggested did help somewhat for FreeBSD. Basically increasing rsize/wsize did help, but he still sees performance below what Linux gets when 9K jumbo frames are used. (I did note the potential problem of mbuf cluster pool fragmentation when 9K jumbo frames are used, although I did not intent to imply that this issue is involved, just that it is a known deficiency that "might" be a factor.) rick > > Best regards > Michael > > > > TL;DR: > > > > * I have a FreeBSD14 Server and Client with an Intel X540 (ix) adaptor = each. > > * I am trying to improve the NFS throughput. > > * I get 1160 MiB/s via nc, but only ~200 MiB/s via NFS. > > * Increasing rsize and wsize to 1 MiB increases throughput to 395 MiB/s > > * But a Linux client achieves 560-600 MiB/s with any rsize. > > * The mtu is set to 9000 but this doesn't make a difference for the Fre= eBSD client (it does make a difference for Linux). > > > > I assume < 400 MiB/s is not the expected performance? Do you have any a= dvice on debugging this? > > > > Thank you for your help, > > Hannes > > > > >