Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 9 Jan 2020 10:57:58 +0100 (CET)
From:      Wojciech Puchar <wojtek@puchar.net>
To:        Conrad Meyer <cem@freebsd.org>
Cc:        Konstantin Belousov <kostikbel@gmail.com>, Hans Petter Selasky <hps@selasky.org>, Rick Macklem <rmacklem@uoguelph.ca>, Wojciech Puchar <wojtek@puchar.net>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
Subject:   Re: maximum MAXBSIZE
Message-ID:  <alpine.BSF.2.20.2001091057420.96836@puchar.net>
In-Reply-To: <CAG6CVpUrGyov12nQSKhofCPw5fAiXgDGChxf3-aFu1fKpirJTQ@mail.gmail.com>
References:  <alpine.BSF.2.20.2001072210410.21107@puchar.net> <d79078c4-f1cb-93b9-ee6e-f689936c1e01@selasky.org> <YQBPR0101MB1427EEDE94AA6E34B49C3C09DD3F0@YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM> <20200108105136.0d54ebce@ernst.home> <alpine.BSF.2.20.2001081452360.44533@puchar.net> <20200108141810.GX23031@kib.kiev.ua> <CAG6CVpUrGyov12nQSKhofCPw5fAiXgDGChxf3-aFu1fKpirJTQ@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
2MB MAXPHYS was what i have set for over 3 years without problems.

On Wed, 8 Jan 2020, Conrad Meyer wrote:

> Bufs are dynamically allocated from uma now, and perhaps a middle ground BSIZE is worth considering? Would 1MB and 2kB bufs (1kB
> 32-bit) be awful?
> 
> Cheers,
> Conrad 
> 
> On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov <kostikbel@gmail.com> wrote:
>       On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote:
>       > sorry i made a mistake - i change MAXPHYS not MAXBSIZE.
>       >
>       > 16MB works for now without problems
>       MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on 32bit).
>
>       >
>       > On Wed, 8 Jan 2020, Gary Jennejohn wrote:
>       >
>       > > On Tue, 7 Jan 2020 22:47:54 +0000
>       > > Rick Macklem <rmacklem@uoguelph.ca> wrote:
>       > >
>       > > > Hans Petter Selasky wrote:
>       > > > > On 2020-01-07 22:12, Wojciech Puchar wrote:
>       > > > > > default MAXBSIZE is 128kB. badly low for todays magnetic disks.
>       > > > > >
>       > > > > > i have it set to 2MB on all computers that have magnetic disks. Great
>       > > > > > improvement with large files. especially when more than one are
>       > > > > > read/wrote in parallel. And no problems experienced
>       > > > > >
>       > > > > > But for optimal performance MAXBSIZE should be transfered in few times
>       > > > > > longer than average seek time. todays disk do 200-250MB/s so 2MB is
>       > > > > > transfered below 10ms.
>       > > > > >
>       > > > > > 8-16MB seems like good choice. is there any reason not to set it that high?
>       > > > >
>       > > > > Old disk may not support it, especially USB 1.0/2.0 disks.
>       > > > I also thought it was limited to MAXPHYS, but maybe I'm only thinking of the NFS
>       > > > specific case?
>       > > >
>       > >
>       > > There's a comment in param.h that it should not exceed MAXPHYS to be
>       > > on the safe side.  How old that comment is I can't say and that may
>       > > not be the case today.
>       > >
>       > > MAXBSIZE is only 64KiB in my param.h.
>       > >
>       > > I have to agree with HPS.  There are many old bridge-chips still in
>       > > use and problems with a large MAXBSIZE might occur.  It's certainly
>       > > not uncommon to see capacity limitations - I have a docking station
>       > > which can't see more than 3TB.
>       > >
>       > > --
>       > > Gary Jennejohn
>       > >
>       > >
>       > _______________________________________________
>       > freebsd-hackers@freebsd.org mailing list
>       > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>       > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>       _______________________________________________
>       freebsd-hackers@freebsd.org mailing list
>       https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>       To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
> 
> 
>
From owner-freebsd-hackers@freebsd.org  Thu Jan  9 14:14:36 2020
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42C941EF6DF
 for <freebsd-hackers@mailman.nyi.freebsd.org>;
 Thu,  9 Jan 2020 14:14:36 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [IPv6:2607:f8b0:4864:20::72e])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 47tp5l2bB0z4BJv
 for <freebsd-hackers@freebsd.org>; Thu,  9 Jan 2020 14:14:35 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qk1-x72e.google.com with SMTP id k6so6060214qki.5
 for <freebsd-hackers@freebsd.org>; Thu, 09 Jan 2020 06:14:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=N/YAohDje5wqg0z4wWLXOqqy27XUFukLKDn7iEXkbiQ=;
 b=rLr0GeSuaeMsCG4DT4+GLhnWqfYrkySkhuC9GWN7bcl3rcsERNz3AfLDeXfAqaqRuh
 3UgziXvG+RcYg5LWEBJYagew5CgR3t9t8gJVjU6mhhqklSjSdWJ4hm1obW/iI2/OHJQf
 CmSyqKcpQK8Xb40CIDsy0/hMGBOjEpDWhcPQEQiyxQomeqqPFg6mPoO6SZLIVOjr4cg4
 0CeGAiGiuQarb7P+umHAdLqq2m98Lj+c3DtsazAHBDUm78qB/fPie6TBIPINHi8ZqIvM
 Q48Aecl2c8loHT2qF6F6sDHO3YuI+FLPScv3BGtPvVDizlv3WPweJmafB9AbByvxlxPs
 DbBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=N/YAohDje5wqg0z4wWLXOqqy27XUFukLKDn7iEXkbiQ=;
 b=A5PAtBuLqSiuL77IgwA8QiLxBikrNCNIeQ3/0GXll9wNlmI3lYbruizIo1jnzUehdI
 muLvQw/ozLYHKDncY62qcFzQ8pra2b1otWTfVLbUsH0Qf/AgtVbm81jw1dkErHcJeegk
 SqH1+lXPpiAgVDZAH01o/Jk7cus/XMW3yLpSIDBVMr3zGUOSLHA2TO0W6i97OPkk6pDE
 b2GoNGvcQu2GuWjLy91cdv8+inTD/Jp7cxcidZacP4zwHsBP42AN6Qi0ihtzWrMI/QpM
 b2tcaZ33lkATBPgJgb/Tt6NdKm2JMBpp2gxAKuhF4nM5cFZ9g5UNEx8dFcRR60UacwMh
 2Lyg==
X-Gm-Message-State: APjAAAWLqmZUxXk6cgUj7sPrIV2WVzaywG5hGm/m4T8BS3d0P+n/kQrX
 dvaNXZPeeL/W1Lp0eJDpw23XRf6FU9iUDQibLWEbUQ==
X-Google-Smtp-Source: APXvYqyEQc+E04/PDle9MMVHLTxltcoJEx+1XLqiRPm4+eegy0pKWFXx5PLipvdHbN6jF/oV1KfLXeGzoIch2T17ni4=
X-Received: by 2002:a37:4f8e:: with SMTP id d136mr9585745qkb.495.1578579273864; 
 Thu, 09 Jan 2020 06:14:33 -0800 (PST)
MIME-Version: 1.0
References: <alpine.BSF.2.20.2001072210410.21107@puchar.net>
 <d79078c4-f1cb-93b9-ee6e-f689936c1e01@selasky.org>
 <YQBPR0101MB1427EEDE94AA6E34B49C3C09DD3F0@YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM>
 <20200108105136.0d54ebce@ernst.home>
 <alpine.BSF.2.20.2001081452360.44533@puchar.net>
 <20200108141810.GX23031@kib.kiev.ua>
 <CAG6CVpUrGyov12nQSKhofCPw5fAiXgDGChxf3-aFu1fKpirJTQ@mail.gmail.com>
 <alpine.BSF.2.20.2001091057420.96836@puchar.net>
In-Reply-To: <alpine.BSF.2.20.2001091057420.96836@puchar.net>
From: Warner Losh <imp@bsdimp.com>
Date: Thu, 9 Jan 2020 07:14:22 -0700
Message-ID: <CANCZdfokuE+KheFvSnx7M4he9Drx31xLj8o_GKUGJqKk32Oj7g@mail.gmail.com>
Subject: Re: maximum MAXBSIZE
To: Wojciech Puchar <wojtek@puchar.net>
Cc: Conrad Meyer <cem@freebsd.org>, Konstantin Belousov <kostikbel@gmail.com>, 
 Hans Petter Selasky <hps@selasky.org>, Rick Macklem <rmacklem@uoguelph.ca>,
 "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
X-Rspamd-Queue-Id: 47tp5l2bB0z4BJv
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=rLr0GeSu; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::72e) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-3.64 / 15.00]; ARC_NA(0.00)[];
 TO_DN_EQ_ADDR_SOME(0.00)[];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[9];
 RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCVD_IN_DNSWL_NONE(0.00)[e.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 MIME_TRACE(0.00)[0:+,1:+,2:~];
 IP_SCORE(-2.65)[ip: (-9.21), ipnet: 2607:f8b0::/32(-2.12), asn: 15169(-1.85),
 country: US(-0.05)]; 
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>;
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Jan 2020 14:14:36 -0000

Netflix runs our entire network at MAXPHYS=8MB since we're doing huge reads
off HDD.

Warner


On Thu, Jan 9, 2020 at 2:58 AM Wojciech Puchar <wojtek@puchar.net> wrote:

> 2MB MAXPHYS was what i have set for over 3 years without problems.
>
> On Wed, 8 Jan 2020, Conrad Meyer wrote:
>
> > Bufs are dynamically allocated from uma now, and perhaps a middle ground
> BSIZE is worth considering? Would 1MB and 2kB bufs (1kB
> > 32-bit) be awful?
> >
> > Cheers,
> > Conrad
> >
> > On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov <kostikbel@gmail.com>
> wrote:
> >       On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote:
> >       > sorry i made a mistake - i change MAXPHYS not MAXBSIZE.
> >       >
> >       > 16MB works for now without problems
> >       MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on
> 32bit).
> >
> >       >
> >       > On Wed, 8 Jan 2020, Gary Jennejohn wrote:
> >       >
> >       > > On Tue, 7 Jan 2020 22:47:54 +0000
> >       > > Rick Macklem <rmacklem@uoguelph.ca> wrote:
> >       > >
> >       > > > Hans Petter Selasky wrote:
> >       > > > > On 2020-01-07 22:12, Wojciech Puchar wrote:
> >       > > > > > default MAXBSIZE is 128kB. badly low for todays magnetic
> disks.
> >       > > > > >
> >       > > > > > i have it set to 2MB on all computers that have magnetic
> disks. Great
> >       > > > > > improvement with large files. especially when more than
> one are
> >       > > > > > read/wrote in parallel. And no problems experienced
> >       > > > > >
> >       > > > > > But for optimal performance MAXBSIZE should be
> transfered in few times
> >       > > > > > longer than average seek time. todays disk do
> 200-250MB/s so 2MB is
> >       > > > > > transfered below 10ms.
> >       > > > > >
> >       > > > > > 8-16MB seems like good choice. is there any reason not
> to set it that high?
> >       > > > >
> >       > > > > Old disk may not support it, especially USB 1.0/2.0 disks.
> >       > > > I also thought it was limited to MAXPHYS, but maybe I'm only
> thinking of the NFS
> >       > > > specific case?
> >       > > >
> >       > >
> >       > > There's a comment in param.h that it should not exceed MAXPHYS
> to be
> >       > > on the safe side.  How old that comment is I can't say and
> that may
> >       > > not be the case today.
> >       > >
> >       > > MAXBSIZE is only 64KiB in my param.h.
> >       > >
> >       > > I have to agree with HPS.  There are many old bridge-chips
> still in
> >       > > use and problems with a large MAXBSIZE might occur.  It's
> certainly
> >       > > not uncommon to see capacity limitations - I have a docking
> station
> >       > > which can't see more than 3TB.
> >       > >
> >       > > --
> >       > > Gary Jennejohn
> >       > >
> >       > >
> >       > _______________________________________________
> >       > freebsd-hackers@freebsd.org mailing list
> >       > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> >       > To unsubscribe, send any mail to "
> freebsd-hackers-unsubscribe@freebsd.org"
> >       _______________________________________________
> >       freebsd-hackers@freebsd.org mailing list
> >       https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> >       To unsubscribe, send any mail to "
> freebsd-hackers-unsubscribe@freebsd.org"
> >
> >
> >
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.20.2001091057420.96836>