From owner-freebsd-questions@freebsd.org Thu Apr 12 19:06:22 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB6C7F87CE7 for ; Thu, 12 Apr 2018 19:06:22 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7709765BE for ; Thu, 12 Apr 2018 19:06:19 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from r56.edvax.de ([92.195.30.67]) by mrelayeu.kundenserver.de (mreue007 [212.227.15.167]) with ESMTPA (Nemesis) id 0MMLX7-1f7X273e6X-0083zu; Thu, 12 Apr 2018 21:06:11 +0200 Date: Thu, 12 Apr 2018 21:06:10 +0200 From: Polytropon To: "Ronald F. Guilmette" Cc: freebsd-questions@freebsd.org Subject: Re: Two questions --- SSD block sizes and buffering Message-Id: <20180412210610.bafc713b.freebsd@edvax.de> In-Reply-To: <23423.1523502187@segfault.tristatelogic.com> References: <23423.1523502187@segfault.tristatelogic.com> Reply-To: Polytropon Organization: EDVAX X-Mailer: Sylpheed 3.1.1 (GTK+ 2.24.5; i386-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:qBUFFIRRqq4iKgn7gxviWOjRgB5fllreGtjq80bhXLeh+eVmeXN RYTbNlgn1qjekG3PAHGU6Kqe0pXQF8h1KNglcN5qD3yQ1Z+v3pQclh4BK5r2WcnYE2rFkms fs0WsZsWZ+ejaKWUfxxHiZsUbCFjDxeHuDdp6PUY9/TEp1Cdcjupht9SQgJXKosTKGz5wdw /lQIWQMK+99b8vYzEqJsg== X-UI-Out-Filterresults: notjunk:1;V01:K0:zDNN9MDd3jI=:wLb00d0iLSZVCqn6hfl9aj ElxUq/xEusixz9i/oAVCC+gxbN+6Oo+jQgl49C/SQJfpMxamHc9OFhhRHWa4CU3oU/4zqOLK0 epBHo4y9DKKI61GOuKwePMnJpDCl8N/sNAO4pzCDxA4tjd6twWrbnE0iyhjbFU+4qI4p51IjQ s9Q4wiVSSUE6eXsPaCL6fHutYhyG5Qnb5GrYQp0j0atYZxk52Zbyr+qLnPRrze1BKOX5KnlSL 5vKmcdZpWbk7hdo4kktgNvF9btg+Wg7j6mUi12Ho9ZVALRw3ZG43Wef5AQ70nlkJ5t6IPmc0h x4Ub/UEsGm18CyZwUlgDUefklvyeAVx5MSpNeEXHK4qASgQHck5GvQDshPC16pAnd9UlmbPHX a40vuFxgfddb90pl39QqvQEhPT9Ji+8lZwH6fYRlo0z0je2ukOsdoEBgYU4PAy3ABrKvkSaor Wv1D+vG4tNS8pWuroJogF5h+NpKe3W7uuuOOvye+MLGChi08WPKo34hYWIBsZhoklliLyJLRX i4pWhM4JqK+GIQ7DtMreioGi+NFtzyjRhd1oTUo0959t1sgf0bNxMlHQfImXNI8L5Ls148UW6 2gK9nYjeKRwoDp5h/vjGiXqTLX5vUu8+CqamqCLIoUoX1jBHuP0truoUpVjEVFA99bALdI2+K UtrNo0nDs1/ga/LTb0VOxVyMR3fvWklEs/7XjyJ0kCrHHTHpMrT4nWKCYAl920zkdKAzr/ccO LdT4ppuMFKCkWcaBa9i72sHhC9I70egoJW8gberrZk9kWtqbK8EDQPcPR9w= X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Apr 2018 19:06:22 -0000 On Wed, 11 Apr 2018 20:03:07 -0700, Ronald F. Guilmette wrote: > > Two rather simple questions: > > 1) I don't know much, generally speaking, but I have certainly read that > the underyling hardware of flash memory products (which I guess > includes both USB sticks and also SSDs) all effectively have > "physical" block sizes on the order of 128 KiB. > > So anyway, I'm just curious to know to what extent, if any, FreeBSD, > when running with, on, or from any such (flash-memory based) mass > storage device does things specifically with these larger physical > block sizes (of flash memory) in mind. No, it just works(TM). :-) > Does FreeBSD automagically sense that it is dealing with an SSD, and > does it then adjust the way it operates on the relevant filesystem(s) > accordingly, for maximal performance? Or must the system administrator > tweek something explicitly (e.g. tunefs parameters, or perhaps newfs > parameters) in order to get optimal performance on/with an SSD? The fragment size is to be applied by the system administrator when initializing the disk, depending on the block size of the device. For example, newfs allows you to do so: # newfs -m 0 -i 16384 -b 16384 -f 4069 -L somelabel \ -t enable -n enable -U /dev/ada0a Note the -f flag; from "man newfs": -f frag-size The fragment size of the file system in bytes. It must be a power of two ranging in value between blocksize/8 and blocksize. The default is 2048 bytes. But as mentioned in many cases: Deviating from the default values should be done for good and educated reasons, and depending on actual requirements. The example above uses a 4k frag size as typically used with newer hard disks. Additionally, slices, partitions and labels should be adjusted to match block size multiples for better performance. > 2) I have written some modest C programs that output lots and lots of > very short little tidbits of data, one per line. In some cases these > programs output millions of lines. > > For reasons that I can explain, these programs explicitly call setvbuf() > on stdout during startup, and they set the buffering type for stdout > to _IOLBF (i.e. line buffering). > > My question is just this: Assme that one of these programs is called > "xyz". Now, if I run the program thusly: > > xyz > xyz.output > > i.e. so that stdout is redirected to a file, will there be one actual > write to disk for each and every line that is written to stdout by xyz? > In other words, will my act of explicitly setting line buffering (for > stdout) in a case like this cause the xyz program to beat the living > hell out of my disk drive? Probably not. The actual write operationg are being issued somewhere "down the line" through the file system driver down to the disk driver. Even a "sync" command issued by the OS will not _immediately_ cause the drive to act. > I hope not, but I'd like to know if it will, or know why it won't, if > it won't. Isn't all output buffered, per default? -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...