Date: Sat, 20 Apr 2019 12:28:22 -0700 (PDT) From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> To: Mark Millard <marklmi@yahoo.com> Cc: Igor Mozolevsky <igor@hybrid-lab.co.uk>, Hackers freeBSD <freebsd-hackers@freebsd.org>, Ian Lepore <ian@freebsd.org> Subject: Re: bhyve VM stopped to boot after moving virtio disks Message-ID: <201904201928.x3KJSMKW023991@gndrsh.dnsmgr.net> In-Reply-To: <E8014037-6A9E-423A-B369-8C06AF024808@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > On 2019-Apr-20, at 10:45, Igor Mozolevsky <igor at hybrid-lab.co.uk> wrote: > > > On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote: > > > > <snip> > >> dd absolutely will fail to copy the last block of the source if it > >> isn't exactly the blocksize and you didn't specify conv=sync, and it > >> will return a zero status when doing so. It appears you've convinced > >> yourself otherwise, but for anyone else reading this thread, be aware: > >> conv=sync is required to copy the last part of the source if it's > >> smaller than the blocksize. > > > > > > Isn't that contrary to the POSIX Spec? Reading the manual [1], it is > > implied (from STDERR part of the main section) that dd will read and > > write partial blocks, cf. truncated blocks?.. > > > > 1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html > > Have you tried as Ian suggests to see if it made a difference? > Ian seems to be speaking FreeBSD, not necessarily POSIX. (I'm > talking actual FreeBSD behavior here, not necessarily documented > behavior.) > > It turns out that https://www.freebsd.org/cgi/man.cgi?dd(1) does say: > > Normally, data resulting from input or conversion or both are aggregated > into output blocks of the specified size. After the end of input is > reached, any remaining output is written as a block. This means that the > final output block may be shorter than the output block size. > > where conv=sync is described as: > > sync > Pad every input block to the input buffer size. Spaces > are used for pad bytes if a block oriented conversion > value is specified, otherwise NUL bytes are used. > > (So sync changes the overall input size when it adds pad bytes, at > least as documented.) > > It appears to me that Ian's expectations of the actual behavior and > the FreeBSD documentation of such are not a match. Absent testing, > I'd bet on Ian, even if it means FreeBSD dd has a bug. So I tested: root@x230a:/tmp/ddtest # dd if=short of=long bs=1m 0+1 records in 0+1 records out 1024 bytes transferred in 0.000101 secs (10171444 bytes/sec) root@x230a:/tmp/ddtest # ls -lag total 62 drwxr-xr-x 2 root wheel 4 Apr 20 19:16 . drwxrwxrwt 44 root wheel 283 Apr 20 19:16 .. -rw-r--r-- 1 root wheel 1024 Apr 20 19:16 long -rw-r--r-- 1 root wheel 1024 Apr 20 19:16 short And on files it is as expected, however the issue Ian describes does infact occur when dealing with block devices that can not do the "truncated" size block operation, which is extremly rare as the block devices are typically either 512, 2048 or 4096 and almost everyone using dd uses block sizes that are large multiples of this so you never end up hitting that edge of trying to write a 1k record to a 4k device. The truncated input record size is garanteed to be a mutliple of the device block size, the one case I can think of that would hit this is dd'ing a 512B disk to a 4096 native disk that happened to NOT be of a size that is a multiple of 4096, something probably VERY rare. In the case of going from a device to a file the chances of lost data are nill (and I have been doing that for 30 years and never hit it.) You DO have to be carefull if your doing forinsics, conv=sync infact can corrupt your forinsics copy in that it is the wrong size and then when someone else does the analysis they get a different results do to this size change. Part of this process usually involves verifying that the size of your image exactly matches the reported size of the device. -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201904201928.x3KJSMKW023991>