From owner-freebsd-hackers@freebsd.org Thu Jan 9 09:58:05 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 55FED22F967 for ; Thu, 9 Jan 2020 09:58:05 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47thPn0X9tz3Nlf; Thu, 9 Jan 2020 09:58:04 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 0099vw07098572 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 9 Jan 2020 10:57:58 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1578563879; bh=5mlkBSP8wWQIIKXpV2AoVhcrtdGOz72iXZ4h7mGIRDk=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=MIb7lzfLTPjzBxGPt1ffkh8qKTzNPBDqpCn6xHpQ7t8R1SmuwI8Lb99T51fCgzprT FE34aCPpG+GmCIxRcb8rbGJClQj8R/7ANdUccMO3wVSnAfkdKX7anbV4LLS/HjnuZa lnAZJL1PNPjSUnS/Sb1QVT3xqRS/Idp0Nf2vJjP8= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 0099vw0H098565; Thu, 9 Jan 2020 10:57:58 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 9 Jan 2020 10:57:58 +0100 (CET) From: Wojciech Puchar To: Conrad Meyer cc: Konstantin Belousov , Hans Petter Selasky , Rick Macklem , Wojciech Puchar , "freebsd-hackers@freebsd.org" Subject: Re: maximum MAXBSIZE In-Reply-To: Message-ID: References: <20200108105136.0d54ebce@ernst.home> <20200108141810.GX23031@kib.kiev.ua> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 X-Rspamd-Queue-Id: 47thPn0X9tz3Nlf X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; REPLY(-4.00)[] Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 09:58:05 -0000 2MB MAXPHYS was what i have set for over 3 years without problems. On Wed, 8 Jan 2020, Conrad Meyer wrote: > Bufs are dynamically allocated from uma now, and perhaps a middle ground BSIZE is worth considering? Would 1MB and 2kB bufs (1kB > 32-bit) be awful? > > Cheers, > Conrad  > > On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov wrote: > On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote: > > sorry i made a mistake - i change MAXPHYS not MAXBSIZE. > > > > 16MB works for now without problems > MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on 32bit). > > > > > On Wed, 8 Jan 2020, Gary Jennejohn wrote: > > > > > On Tue, 7 Jan 2020 22:47:54 +0000 > > > Rick Macklem wrote: > > > > > > > Hans Petter Selasky wrote: > > > > > On 2020-01-07 22:12, Wojciech Puchar wrote: > > > > > > default MAXBSIZE is 128kB. badly low for todays magnetic disks. > > > > > > > > > > > > i have it set to 2MB on all computers that have magnetic disks. Great > > > > > > improvement with large files. especially when more than one are > > > > > > read/wrote in parallel. And no problems experienced > > > > > > > > > > > > But for optimal performance MAXBSIZE should be transfered in few times > > > > > > longer than average seek time. todays disk do 200-250MB/s so 2MB is > > > > > > transfered below 10ms. > > > > > > > > > > > > 8-16MB seems like good choice. is there any reason not to set it that high? > > > > > > > > > > Old disk may not support it, especially USB 1.0/2.0 disks. > > > > I also thought it was limited to MAXPHYS, but maybe I'm only thinking of the NFS > > > > specific case? > > > > > > > > > > There's a comment in param.h that it should not exceed MAXPHYS to be > > > on the safe side.  How old that comment is I can't say and that may > > > not be the case today. > > > > > > MAXBSIZE is only 64KiB in my param.h. > > > > > > I have to agree with HPS.  There are many old bridge-chips still in > > > use and problems with a large MAXBSIZE might occur.  It's certainly > > > not uncommon to see capacity limitations - I have a docking station > > > which can't see more than 3TB. > > > > > > -- > > > Gary Jennejohn > > > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-hackers@freebsd.org Thu Jan 9 14:14:36 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42C941EF6DF for ; Thu, 9 Jan 2020 14:14:36 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47tp5l2bB0z4BJv for ; Thu, 9 Jan 2020 14:14:35 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72e.google.com with SMTP id k6so6060214qki.5 for ; Thu, 09 Jan 2020 06:14:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=N/YAohDje5wqg0z4wWLXOqqy27XUFukLKDn7iEXkbiQ=; b=rLr0GeSuaeMsCG4DT4+GLhnWqfYrkySkhuC9GWN7bcl3rcsERNz3AfLDeXfAqaqRuh 3UgziXvG+RcYg5LWEBJYagew5CgR3t9t8gJVjU6mhhqklSjSdWJ4hm1obW/iI2/OHJQf CmSyqKcpQK8Xb40CIDsy0/hMGBOjEpDWhcPQEQiyxQomeqqPFg6mPoO6SZLIVOjr4cg4 0CeGAiGiuQarb7P+umHAdLqq2m98Lj+c3DtsazAHBDUm78qB/fPie6TBIPINHi8ZqIvM Q48Aecl2c8loHT2qF6F6sDHO3YuI+FLPScv3BGtPvVDizlv3WPweJmafB9AbByvxlxPs DbBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=N/YAohDje5wqg0z4wWLXOqqy27XUFukLKDn7iEXkbiQ=; b=A5PAtBuLqSiuL77IgwA8QiLxBikrNCNIeQ3/0GXll9wNlmI3lYbruizIo1jnzUehdI muLvQw/ozLYHKDncY62qcFzQ8pra2b1otWTfVLbUsH0Qf/AgtVbm81jw1dkErHcJeegk SqH1+lXPpiAgVDZAH01o/Jk7cus/XMW3yLpSIDBVMr3zGUOSLHA2TO0W6i97OPkk6pDE b2GoNGvcQu2GuWjLy91cdv8+inTD/Jp7cxcidZacP4zwHsBP42AN6Qi0ihtzWrMI/QpM b2tcaZ33lkATBPgJgb/Tt6NdKm2JMBpp2gxAKuhF4nM5cFZ9g5UNEx8dFcRR60UacwMh 2Lyg== X-Gm-Message-State: APjAAAWLqmZUxXk6cgUj7sPrIV2WVzaywG5hGm/m4T8BS3d0P+n/kQrX dvaNXZPeeL/W1Lp0eJDpw23XRf6FU9iUDQibLWEbUQ== X-Google-Smtp-Source: APXvYqyEQc+E04/PDle9MMVHLTxltcoJEx+1XLqiRPm4+eegy0pKWFXx5PLipvdHbN6jF/oV1KfLXeGzoIch2T17ni4= X-Received: by 2002:a37:4f8e:: with SMTP id d136mr9585745qkb.495.1578579273864; Thu, 09 Jan 2020 06:14:33 -0800 (PST) MIME-Version: 1.0 References: <20200108105136.0d54ebce@ernst.home> <20200108141810.GX23031@kib.kiev.ua> In-Reply-To: From: Warner Losh Date: Thu, 9 Jan 2020 07:14:22 -0700 Message-ID: Subject: Re: maximum MAXBSIZE To: Wojciech Puchar Cc: Conrad Meyer , Konstantin Belousov , Hans Petter Selasky , Rick Macklem , "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 47tp5l2bB0z4BJv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=rLr0GeSu; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.64 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[9]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[e.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.65)[ip: (-9.21), ipnet: 2607:f8b0::/32(-2.12), asn: 15169(-1.85), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 14:14:36 -0000 Netflix runs our entire network at MAXPHYS=8MB since we're doing huge reads off HDD. Warner On Thu, Jan 9, 2020 at 2:58 AM Wojciech Puchar wrote: > 2MB MAXPHYS was what i have set for over 3 years without problems. > > On Wed, 8 Jan 2020, Conrad Meyer wrote: > > > Bufs are dynamically allocated from uma now, and perhaps a middle ground > BSIZE is worth considering? Would 1MB and 2kB bufs (1kB > > 32-bit) be awful? > > > > Cheers, > > Conrad > > > > On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov > wrote: > > On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote: > > > sorry i made a mistake - i change MAXPHYS not MAXBSIZE. > > > > > > 16MB works for now without problems > > MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on > 32bit). > > > > > > > > On Wed, 8 Jan 2020, Gary Jennejohn wrote: > > > > > > > On Tue, 7 Jan 2020 22:47:54 +0000 > > > > Rick Macklem wrote: > > > > > > > > > Hans Petter Selasky wrote: > > > > > > On 2020-01-07 22:12, Wojciech Puchar wrote: > > > > > > > default MAXBSIZE is 128kB. badly low for todays magnetic > disks. > > > > > > > > > > > > > > i have it set to 2MB on all computers that have magnetic > disks. Great > > > > > > > improvement with large files. especially when more than > one are > > > > > > > read/wrote in parallel. And no problems experienced > > > > > > > > > > > > > > But for optimal performance MAXBSIZE should be > transfered in few times > > > > > > > longer than average seek time. todays disk do > 200-250MB/s so 2MB is > > > > > > > transfered below 10ms. > > > > > > > > > > > > > > 8-16MB seems like good choice. is there any reason not > to set it that high? > > > > > > > > > > > > Old disk may not support it, especially USB 1.0/2.0 disks. > > > > > I also thought it was limited to MAXPHYS, but maybe I'm only > thinking of the NFS > > > > > specific case? > > > > > > > > > > > > > There's a comment in param.h that it should not exceed MAXPHYS > to be > > > > on the safe side. How old that comment is I can't say and > that may > > > > not be the case today. > > > > > > > > MAXBSIZE is only 64KiB in my param.h. > > > > > > > > I have to agree with HPS. There are many old bridge-chips > still in > > > > use and problems with a large MAXBSIZE might occur. It's > certainly > > > > not uncommon to see capacity limitations - I have a docking > station > > > > which can't see more than 3TB. > > > > > > > > -- > > > > Gary Jennejohn > > > > > > > > > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >