Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 14 Apr 2018 01:06:36 +0200
From:      Polytropon <freebsd@edvax.de>
To:        Arthur Chance <freebsd@qeng-ho.org>
Cc:        "Ronald F. Guilmette" <rfg@tristatelogic.com>, freebsd-questions@freebsd.org
Subject:   Re: Two questions --- SSD block sizes and buffering
Message-ID:  <20180414010636.18b4e568.freebsd@edvax.de>
In-Reply-To: <1cc45af6-bd4b-3854-4d37-8e9343786ce6@qeng-ho.org>
References:  <23423.1523502187@segfault.tristatelogic.com> <20180412210610.bafc713b.freebsd@edvax.de> <1cc45af6-bd4b-3854-4d37-8e9343786ce6@qeng-ho.org>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
On Fri, 13 Apr 2018 08:27:14 +0100, Arthur Chance wrote:
> On 12/04/2018 20:06, Polytropon wrote:
> > On Wed, 11 Apr 2018 20:03:07 -0700, Ronald F. Guilmette wrote:
> >>
> >> Two rather simple questions:
> >>
> >> 1)  I don't know much, generally speaking, but I have certainly read that
> >>     the underyling hardware of flash memory products (which I guess
> >>     includes both USB sticks and also SSDs) all effectively have
> >>     "physical" block sizes on the order of 128 KiB.
> >>     
> >>     So anyway, I'm just curious to know to what extent, if any, FreeBSD,
> >>     when running with, on, or from any such (flash-memory based) mass
> >>     storage device does things specifically with these larger physical
> >>     block sizes (of flash memory) in mind.
> > 
> > No, it just works(TM). :-)
> > 
> > 
> > 
> >>     Does FreeBSD automagically sense that it is dealing with an SSD, and
> >>     does it then adjust the way it operates on the relevant filesystem(s)
> >>     accordingly, for maximal performance?  Or must the system administrator
> >>     tweek something explicitly (e.g. tunefs parameters, or perhaps newfs
> >>     parameters) in order to get optimal performance on/with an SSD?
> > 
> > The fragment size is to be applied by the system administrator
> > when initializing the disk, depending on the block size of the
> > device. For example, newfs allows you to do so:
> > 
> > 	# newfs -m 0 -i 16384 -b 16384 -f 4069 -L somelabel	\
> > 	  -t enable -n enable -U /dev/ada0a
> > 
> > Note the -f flag; from "man newfs":
> > 
> >      -f frag-size
> >              The fragment size of the file system in bytes.  It must be a
> >              power of two ranging in value between blocksize/8 and blocksize.
> >              The default is 2048 bytes.
> 
> Which revision of FBSD are you running? I'm on 10 (must upgrade) and man
> newfs says
> 
> -f frag-size
>         The fragment size of the file system in bytes.  It must be a
>         power of two ranging in value between blocksize/8 and blocksize.
>         The default is 4096 bytes.
> 
> so it's been fixed for 4k disks.

Yes, this has been taken from a 8.x system I was just logged in to.
You know, one of those systems that run and run and run and run and
run and run... obeying the "never touch a running system" rule. :-)

You are correct, 4k became the default frag size, so adjusting it
manually is only needed for specific cases now.



> > But as mentioned in many cases: Deviating from the default
> > values should be done for good and educated reasons, and
> > depending on actual requirements.
> > 
> > The example above uses a 4k frag size as typically used
> > with newer hard disks. Additionally, slices, partitions
> > and labels should be adjusted to match block size multiples
> > for better performance.
> 
> It's worth noting that by default the first GPT partition (which is
> usually the boot partition) starts at block 34, which isn't a multiple
> of 4k. I always set it to start at block 40 to align with 4k disks and
> start the second partition at block 2048 so it's 1 MB aligned.

Fully agree. Using 1M as factor makes it easy to align the
partitions.



> >> 2)  I have written some modest C programs that output lots and lots of
> >>     very short little tidbits of data, one per line.  In some cases these
> >>     programs output millions of lines.
> >>
> >>     For reasons that I can explain, these programs explicitly call setvbuf()
> >>     on stdout during startup, and they set the buffering type for stdout
> >>     to _IOLBF (i.e. line buffering).
> >>
> >>     My question is just this:  Assme that one of these programs is called
> >>     "xyz".  Now, if I run the program thusly:
> >>
> >>            xyz > xyz.output
> >>
> >>      i.e. so that stdout is redirected to a file, will there be one actual
> >>      write to disk for each and every line that is written to stdout by xyz?
> >>      In other words, will my act of explicitly setting line buffering (for
> >>      stdout) in a case like this cause the xyz program to beat the living
> >>      hell out of my disk drive?
> > 
> > Probably not. The actual write operationg are being issued
> > somewhere "down the line" through the file system driver
> > down to the disk driver. Even a "sync" command issued by
> > the OS will not _immediately_ cause the drive to act.
> 
> write(2) calls, which underlie stdio, add the data to the disk block
> image in the VM cache. Unless your machine is under extreme memory
> pressure or you call fsync or sync the buffers eventually get written
> out by a kernel task. See man syncer for details.
> 
> >>      I hope not, but I'd like to know if it will, or know why it won't, if
> >>      it won't.
> > 
> > Isn't all output buffered, per default?
> 
> There is a magic open flag that prevents it, but that's for those doing
> raw block io to devices.

Interesting pointer, thanks! I think I found it in "man 2 open". :-)




-- 
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?20180414010636.18b4e568.freebsd>