From owner-freebsd-questions@freebsd.org Fri Apr 13 07:33:44 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 349A9F96520 for ; Fri, 13 Apr 2018 07:33:44 +0000 (UTC) (envelope-from freebsd@qeng-ho.org) Received: from bede.qeng-ho.org (bede.qeng-ho.org [217.155.128.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 842166EB43 for ; Fri, 13 Apr 2018 07:33:43 +0000 (UTC) (envelope-from freebsd@qeng-ho.org) Received: from arthur.home.qeng-ho.org (arthur.home.qeng-ho.org [172.23.1.2]) by bede.qeng-ho.org (Postfix) with ESMTP id 6BB1110377; Fri, 13 Apr 2018 08:27:14 +0100 (BST) Subject: Re: Two questions --- SSD block sizes and buffering To: Polytropon , "Ronald F. Guilmette" Cc: freebsd-questions@freebsd.org References: <23423.1523502187@segfault.tristatelogic.com> <20180412210610.bafc713b.freebsd@edvax.de> From: Arthur Chance Message-ID: <1cc45af6-bd4b-3854-4d37-8e9343786ce6@qeng-ho.org> Date: Fri, 13 Apr 2018 08:27:14 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180412210610.bafc713b.freebsd@edvax.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Apr 2018 07:33:44 -0000 On 12/04/2018 20:06, Polytropon wrote: > On Wed, 11 Apr 2018 20:03:07 -0700, Ronald F. Guilmette wrote: >> >> Two rather simple questions: >> >> 1) I don't know much, generally speaking, but I have certainly read that >> the underyling hardware of flash memory products (which I guess >> includes both USB sticks and also SSDs) all effectively have >> "physical" block sizes on the order of 128 KiB. >> >> So anyway, I'm just curious to know to what extent, if any, FreeBSD, >> when running with, on, or from any such (flash-memory based) mass >> storage device does things specifically with these larger physical >> block sizes (of flash memory) in mind. > > No, it just works(TM). :-) > > > >> Does FreeBSD automagically sense that it is dealing with an SSD, and >> does it then adjust the way it operates on the relevant filesystem(s) >> accordingly, for maximal performance? Or must the system administrator >> tweek something explicitly (e.g. tunefs parameters, or perhaps newfs >> parameters) in order to get optimal performance on/with an SSD? > > The fragment size is to be applied by the system administrator > when initializing the disk, depending on the block size of the > device. For example, newfs allows you to do so: > > # newfs -m 0 -i 16384 -b 16384 -f 4069 -L somelabel \ > -t enable -n enable -U /dev/ada0a > > Note the -f flag; from "man newfs": > > -f frag-size > The fragment size of the file system in bytes. It must be a > power of two ranging in value between blocksize/8 and blocksize. > The default is 2048 bytes. Which revision of FBSD are you running? I'm on 10 (must upgrade) and man newfs says -f frag-size The fragment size of the file system in bytes. It must be a power of two ranging in value between blocksize/8 and blocksize. The default is 4096 bytes. so it's been fixed for 4k disks. > But as mentioned in many cases: Deviating from the default > values should be done for good and educated reasons, and > depending on actual requirements. > > The example above uses a 4k frag size as typically used > with newer hard disks. Additionally, slices, partitions > and labels should be adjusted to match block size multiples > for better performance. It's worth noting that by default the first GPT partition (which is usually the boot partition) starts at block 34, which isn't a multiple of 4k. I always set it to start at block 40 to align with 4k disks and start the second partition at block 2048 so it's 1 MB aligned. > >> 2) I have written some modest C programs that output lots and lots of >> very short little tidbits of data, one per line. In some cases these >> programs output millions of lines. >> >> For reasons that I can explain, these programs explicitly call setvbuf() >> on stdout during startup, and they set the buffering type for stdout >> to _IOLBF (i.e. line buffering). >> >> My question is just this: Assme that one of these programs is called >> "xyz". Now, if I run the program thusly: >> >> xyz > xyz.output >> >> i.e. so that stdout is redirected to a file, will there be one actual >> write to disk for each and every line that is written to stdout by xyz? >> In other words, will my act of explicitly setting line buffering (for >> stdout) in a case like this cause the xyz program to beat the living >> hell out of my disk drive? > > Probably not. The actual write operationg are being issued > somewhere "down the line" through the file system driver > down to the disk driver. Even a "sync" command issued by > the OS will not _immediately_ cause the drive to act. write(2) calls, which underlie stdio, add the data to the disk block image in the VM cache. Unless your machine is under extreme memory pressure or you call fsync or sync the buffers eventually get written out by a kernel task. See man syncer for details. >> I hope not, but I'd like to know if it will, or know why it won't, if >> it won't. > > Isn't all output buffered, per default? There is a magic open flag that prevents it, but that's for those doing raw block io to devices. -- An amusing coincidence: log2(58) = 5.858 (to 0.0003% accuracy).