From owner-freebsd-hackers@freebsd.org Thu Jan 9 14:21:30 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 876391EFB12 for ; Thu, 9 Jan 2020 14:21:30 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47tpFj2T78z4Byx for ; Thu, 9 Jan 2020 14:21:29 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 009ELPjE011261 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 9 Jan 2020 15:21:25 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1578579686; bh=LTGsn+vClnvordGRbo+urfNOwDwu0JSz/jnVn1OoBHU=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=Yedg4vsUQDJ6HRL6wKEIweSvsrSY5aGUDeELSUFYLgG7QCMReQLh9YPJg4CHI3SwS UVny5T6c2QquVtuTtRh8EeP6VBhQpMqxUVoIVxRAvqCUmz2I3PSJgng5O+QYqYlyaM CqYAl1q7NEsTnDOHfGqvBODqN5dv810p4FWxdA0c= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 009ELPtX011258; Thu, 9 Jan 2020 15:21:25 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 9 Jan 2020 15:21:25 +0100 (CET) From: Wojciech Puchar To: Warner Losh cc: Wojciech Puchar , Conrad Meyer , Konstantin Belousov , Hans Petter Selasky , Rick Macklem , "freebsd-hackers@freebsd.org" Subject: Re: maximum MAXBSIZE In-Reply-To: Message-ID: References: <20200108105136.0d54ebce@ernst.home> <20200108141810.GX23031@kib.kiev.ua> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 X-Rspamd-Queue-Id: 47tpFj2T78z4Byx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=Yedg4vsU; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.33 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; CTYPE_MIXED_BOGUS(1.00)[]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; RCPT_COUNT_SEVEN(0.00)[7]; IP_SCORE(-3.04)[ip: (-8.03), ipnet: 194.1.144.0/24(-4.01), asn: 43476(-3.21), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 14:21:30 -0000 why FreeBSD default is so completely wrong for modern hardware? i think 4MB is OK for HDDs, more may be optimal for RAID5 arrays. On Thu, 9 Jan 2020, Warner Losh wrote: > Netflix runs our entire network at MAXPHYS=8MB since we're doing huge reads off HDD. > Warner > > > On Thu, Jan 9, 2020 at 2:58 AM Wojciech Puchar wrote: > 2MB MAXPHYS was what i have set for over 3 years without problems. > > On Wed, 8 Jan 2020, Conrad Meyer wrote: > > > Bufs are dynamically allocated from uma now, and perhaps a middle ground BSIZE is worth considering? Would 1MB and 2kB > bufs (1kB > > 32-bit) be awful? > > > > Cheers, > > Conrad  > > > > On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov wrote: > >       On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote: > >       > sorry i made a mistake - i change MAXPHYS not MAXBSIZE. > >       > > >       > 16MB works for now without problems > >       MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on 32bit). > > > >       > > >       > On Wed, 8 Jan 2020, Gary Jennejohn wrote: > >       > > >       > > On Tue, 7 Jan 2020 22:47:54 +0000 > >       > > Rick Macklem wrote: > >       > > > >       > > > Hans Petter Selasky wrote: > >       > > > > On 2020-01-07 22:12, Wojciech Puchar wrote: > >       > > > > > default MAXBSIZE is 128kB. badly low for todays magnetic disks. > >       > > > > > > >       > > > > > i have it set to 2MB on all computers that have magnetic disks. Great > >       > > > > > improvement with large files. especially when more than one are > >       > > > > > read/wrote in parallel. And no problems experienced > >       > > > > > > >       > > > > > But for optimal performance MAXBSIZE should be transfered in few times > >       > > > > > longer than average seek time. todays disk do 200-250MB/s so 2MB is > >       > > > > > transfered below 10ms. > >       > > > > > > >       > > > > > 8-16MB seems like good choice. is there any reason not to set it that high? > >       > > > > > >       > > > > Old disk may not support it, especially USB 1.0/2.0 disks. > >       > > > I also thought it was limited to MAXPHYS, but maybe I'm only thinking of the NFS > >       > > > specific case? > >       > > > > >       > > > >       > > There's a comment in param.h that it should not exceed MAXPHYS to be > >       > > on the safe side.  How old that comment is I can't say and that may > >       > > not be the case today. > >       > > > >       > > MAXBSIZE is only 64KiB in my param.h. > >       > > > >       > > I have to agree with HPS.  There are many old bridge-chips still in > >       > > use and problems with a large MAXBSIZE might occur.  It's certainly > >       > > not uncommon to see capacity limitations - I have a docking station > >       > > which can't see more than 3TB. > >       > > > >       > > -- > >       > > Gary Jennejohn > >       > > > >       > > > >       > _______________________________________________ > >       > freebsd-hackers@freebsd.org mailing list > >       > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >       > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > >       _______________________________________________ > >       freebsd-hackers@freebsd.org mailing list > >       https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >       To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-hackers@freebsd.org Thu Jan 9 15:45:25 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DE8B71F1957 for ; Thu, 9 Jan 2020 15:45:25 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47tr6Y2njBz4JXG; Thu, 9 Jan 2020 15:45:25 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wr1-x431.google.com with SMTP id j42so7829979wrj.12; Thu, 09 Jan 2020 07:45:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=mfO3QDVmZmHLUvb1DXfLFUXZrWM2ViERfKfEzC411ks=; b=YE7G2p9KLvLRGjZImDcA4icpgnDS7Qt27Cg9AOpEi6xHQaVexsM7hiVgCRhmPBJ6VG EhBR/siizp/N4ymOW6yO+acXsaFqQzxMaxjnEwu9sGTwIcGNS3lRYC0o3/alSvZy3FO1 dHW+BKZkpqmOfOhrqCTQTnAD672xJcVSmv3foXJGYLWapOE/70ec6PYXd4VSL5IQJ1Lg 9Mn67w05NP3kZq5epMPa2fFL07Z87BD24bZe9IljzojGawc6rBlq4YWwikkvR9lxIfdz Ei5diYNVkKxsoJDhFLXlZ9yJYf+mjBEcd2DyrKdpTnb7CrW79ipRjoyAPdo/X+n2Q90K ubYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=mfO3QDVmZmHLUvb1DXfLFUXZrWM2ViERfKfEzC411ks=; b=DsjjcBp1NRpCjaIPqbudDhqjKvxCj5QNjv2eNpcKMQcoMQC8tDtQOHN24yTtfzqRol 2tklHWupnoD3GUHHtWJF5xrM7RQdG9wT5hyayQP/WKZLXZhpMHkgatVk+ZAqqNsdOmOV Vll+Du9uimb0ggcNUsTtuflFL6YAsaEw5xdDfZWQnrIH1rLyeeOLx7VqnkRNZfycPnkn QbXRpSEYspGXvMvP4ehwvQFkqS+MEiGnHW6uGkurHc6GU37O0n52/AD2m5U5rWgkSlxV tXE06G08IOwHgtDTPIa5oVqRg2wTzdwNow3T5+OyD7J3yerWTAoDv65D3NyhTRi9mv5j 227g== X-Gm-Message-State: APjAAAXumje2WX8riSHBhie8wplGLyaTwIrCQ47Cgvy/dimKnIlgPsIH 5rIxrHfk7L+x+fQ419RU1gs= X-Google-Smtp-Source: APXvYqzj/JRyUVaKhY6yB1xIgpP7W1PXL71jkLjuHA+936xuoBNX/7ELk+eOUzD7TymGdgnxFcJwgA== X-Received: by 2002:a5d:53d1:: with SMTP id a17mr11188284wrw.327.1578584723753; Thu, 09 Jan 2020 07:45:23 -0800 (PST) Received: from ernst.home (pD9E2375C.dip0.t-ipconnect.de. [217.226.55.92]) by smtp.gmail.com with ESMTPSA id u22sm8553961wru.30.2020.01.09.07.45.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 07:45:23 -0800 (PST) Date: Thu, 9 Jan 2020 16:45:19 +0100 From: Gary Jennejohn To: Wojciech Puchar Cc: Warner Losh , Hans Petter Selasky , Rick Macklem , Conrad Meyer , "freebsd-hackers@freebsd.org" , Konstantin Belousov Subject: Re: maximum MAXBSIZE Message-ID: <20200109164519.33fc7478@ernst.home> In-Reply-To: References: <20200108105136.0d54ebce@ernst.home> <20200108141810.GX23031@kib.kiev.ua> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47tr6Y2njBz4JXG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=YE7G2p9K; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::431 as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCPT_COUNT_SEVEN(0.00)[7]; RECEIVED_SPAMHAUS_PBL(0.00)[92.55.226.217.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (-9.05), ipnet: 2a00:1450::/32(-2.61), asn: 15169(-1.85), country: US(-0.05)]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 15:45:25 -0000 On Thu, 9 Jan 2020 15:21:25 +0100 (CET) Wojciech Puchar wrote: > why FreeBSD default is so completely wrong for modern hardware? > > i think 4MB is OK for HDDs, more may be optimal for RAID5 arrays. > POLA (principle of least amazement). I certainly don't need a MAXPHYS set to 4MB on my desktop machine. Not everyone using FreeBSD is running servers with large amounts of memory and disk storage. It's a trivial change if it's beneficial in a certain use scenario. The decision should be left up to the user. > > On Thu, 9 Jan 2020, Warner Losh wrote: > > > Netflix runs our entire network at MAXPHYS=8MB since we're doing huge reads off HDD. > > Warner > > > > > > On Thu, Jan 9, 2020 at 2:58 AM Wojciech Puchar wrote: > > 2MB MAXPHYS was what i have set for over 3 years without problems. > > > > On Wed, 8 Jan 2020, Conrad Meyer wrote: > > > > > Bufs are dynamically allocated from uma now, and perhaps a middle ground BSIZE is worth considering? Would 1MB and 2kB > > bufs (1kB > > > 32-bit) be awful? > > > > > > Cheers, > > > Conrad__ > > > > > > On Wed, Jan 8, 2020 at 06:18 Konstantin Belousov wrote: > > >__ __ __ __On Wed, Jan 08, 2020 at 02:52:57PM +0100, Wojciech Puchar wrote: > > >__ __ __ __> sorry i made a mistake - i change MAXPHYS not MAXBSIZE. > > >__ __ __ __> > > >__ __ __ __> 16MB works for now without problems > > >__ __ __ __MAXPHYS 16MB means that sizeof(struct buf) is around 32K (16K on 32bit). > > > > > >__ __ __ __> > > >__ __ __ __> On Wed, 8 Jan 2020, Gary Jennejohn wrote: > > >__ __ __ __> > > >__ __ __ __> > On Tue, 7 Jan 2020 22:47:54 +0000 > > >__ __ __ __> > Rick Macklem wrote: > > >__ __ __ __> > > > >__ __ __ __> > > Hans Petter Selasky wrote: > > >__ __ __ __> > > > On 2020-01-07 22:12, Wojciech Puchar wrote: > > >__ __ __ __> > > > > default MAXBSIZE is 128kB. badly low for todays magnetic disks. > > >__ __ __ __> > > > > > > >__ __ __ __> > > > > i have it set to 2MB on all computers that have magnetic disks. Great > > >__ __ __ __> > > > > improvement with large files. especially when more than one are > > >__ __ __ __> > > > > read/wrote in parallel. And no problems experienced > > >__ __ __ __> > > > > > > >__ __ __ __> > > > > But for optimal performance MAXBSIZE should be transfered in few times > > >__ __ __ __> > > > > longer than average seek time. todays disk do 200-250MB/s so 2MB is > > >__ __ __ __> > > > > transfered below 10ms. > > >__ __ __ __> > > > > > > >__ __ __ __> > > > > 8-16MB seems like good choice. is there any reason not to set it that high? > > >__ __ __ __> > > > > > >__ __ __ __> > > > Old disk may not support it, especially USB 1.0/2.0 disks. > > >__ __ __ __> > > I also thought it was limited to MAXPHYS, but maybe I'm only thinking of the NFS > > >__ __ __ __> > > specific case? > > >__ __ __ __> > > > > >__ __ __ __> > > > >__ __ __ __> > There's a comment in param.h that it should not exceed MAXPHYS to be > > >__ __ __ __> > on the safe side.__ How old that comment is I can't say and that may > > >__ __ __ __> > not be the case today. > > >__ __ __ __> > > > >__ __ __ __> > MAXBSIZE is only 64KiB in my param.h. > > >__ __ __ __> > > > >__ __ __ __> > I have to agree with HPS.__ There are many old bridge-chips still in > > >__ __ __ __> > use and problems with a large MAXBSIZE might occur.__ It's certainly > > >__ __ __ __> > not uncommon to see capacity limitations - I have a docking station > > >__ __ __ __> > which can't see more than 3TB. > > >__ __ __ __> > > > >__ __ __ __> > -- > > >__ __ __ __> > Gary Jennejohn > > >__ __ __ __> > > > >__ __ __ __> > > > >__ __ __ __> _______________________________________________ > > >__ __ __ __> freebsd-hackers@freebsd.org mailing list > > >__ __ __ __> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > >__ __ __ __> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > >__ __ __ _________________________________________________ > > >__ __ __ __freebsd-hackers@freebsd.org mailing list > > >__ __ __ __https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > >__ __ __ __To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Gary Jennejohn