Date: Sat, 14 Apr 2018 01:06:36 +0200 From: Polytropon <freebsd@edvax.de> To: Arthur Chance <freebsd@qeng-ho.org> Cc: "Ronald F. Guilmette" <rfg@tristatelogic.com>, freebsd-questions@freebsd.org Subject: Re: Two questions --- SSD block sizes and buffering Message-ID: <20180414010636.18b4e568.freebsd@edvax.de> In-Reply-To: <1cc45af6-bd4b-3854-4d37-8e9343786ce6@qeng-ho.org> References: <23423.1523502187@segfault.tristatelogic.com> <20180412210610.bafc713b.freebsd@edvax.de> <1cc45af6-bd4b-3854-4d37-8e9343786ce6@qeng-ho.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 13 Apr 2018 08:27:14 +0100, Arthur Chance wrote: > On 12/04/2018 20:06, Polytropon wrote: > > On Wed, 11 Apr 2018 20:03:07 -0700, Ronald F. Guilmette wrote: > >> > >> Two rather simple questions: > >> > >> 1) I don't know much, generally speaking, but I have certainly read that > >> the underyling hardware of flash memory products (which I guess > >> includes both USB sticks and also SSDs) all effectively have > >> "physical" block sizes on the order of 128 KiB. > >> > >> So anyway, I'm just curious to know to what extent, if any, FreeBSD, > >> when running with, on, or from any such (flash-memory based) mass > >> storage device does things specifically with these larger physical > >> block sizes (of flash memory) in mind. > > > > No, it just works(TM). :-) > > > > > > > >> Does FreeBSD automagically sense that it is dealing with an SSD, and > >> does it then adjust the way it operates on the relevant filesystem(s) > >> accordingly, for maximal performance? Or must the system administrator > >> tweek something explicitly (e.g. tunefs parameters, or perhaps newfs > >> parameters) in order to get optimal performance on/with an SSD? > > > > The fragment size is to be applied by the system administrator > > when initializing the disk, depending on the block size of the > > device. For example, newfs allows you to do so: > > > > # newfs -m 0 -i 16384 -b 16384 -f 4069 -L somelabel \ > > -t enable -n enable -U /dev/ada0a > > > > Note the -f flag; from "man newfs": > > > > -f frag-size > > The fragment size of the file system in bytes. It must be a > > power of two ranging in value between blocksize/8 and blocksize. > > The default is 2048 bytes. > > Which revision of FBSD are you running? I'm on 10 (must upgrade) and man > newfs says > > -f frag-size > The fragment size of the file system in bytes. It must be a > power of two ranging in value between blocksize/8 and blocksize. > The default is 4096 bytes. > > so it's been fixed for 4k disks. Yes, this has been taken from a 8.x system I was just logged in to. You know, one of those systems that run and run and run and run and run and run... obeying the "never touch a running system" rule. :-) You are correct, 4k became the default frag size, so adjusting it manually is only needed for specific cases now. > > But as mentioned in many cases: Deviating from the default > > values should be done for good and educated reasons, and > > depending on actual requirements. > > > > The example above uses a 4k frag size as typically used > > with newer hard disks. Additionally, slices, partitions > > and labels should be adjusted to match block size multiples > > for better performance. > > It's worth noting that by default the first GPT partition (which is > usually the boot partition) starts at block 34, which isn't a multiple > of 4k. I always set it to start at block 40 to align with 4k disks and > start the second partition at block 2048 so it's 1 MB aligned. Fully agree. Using 1M as factor makes it easy to align the partitions. > >> 2) I have written some modest C programs that output lots and lots of > >> very short little tidbits of data, one per line. In some cases these > >> programs output millions of lines. > >> > >> For reasons that I can explain, these programs explicitly call setvbuf() > >> on stdout during startup, and they set the buffering type for stdout > >> to _IOLBF (i.e. line buffering). > >> > >> My question is just this: Assme that one of these programs is called > >> "xyz". Now, if I run the program thusly: > >> > >> xyz > xyz.output > >> > >> i.e. so that stdout is redirected to a file, will there be one actual > >> write to disk for each and every line that is written to stdout by xyz? > >> In other words, will my act of explicitly setting line buffering (for > >> stdout) in a case like this cause the xyz program to beat the living > >> hell out of my disk drive? > > > > Probably not. The actual write operationg are being issued > > somewhere "down the line" through the file system driver > > down to the disk driver. Even a "sync" command issued by > > the OS will not _immediately_ cause the drive to act. > > write(2) calls, which underlie stdio, add the data to the disk block > image in the VM cache. Unless your machine is under extreme memory > pressure or you call fsync or sync the buffers eventually get written > out by a kernel task. See man syncer for details. > > >> I hope not, but I'd like to know if it will, or know why it won't, if > >> it won't. > > > > Isn't all output buffered, per default? > > There is a magic open flag that prevents it, but that's for those doing > raw block io to devices. Interesting pointer, thanks! I think I found it in "man 2 open". :-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180414010636.18b4e568.freebsd>