From owner-freebsd-questions@freebsd.org Sun Dec 1 21:09:41 2019 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 588681B9575 for ; Sun, 1 Dec 2019 21:09:41 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from mx.sdf.org (mx.sdf.org [205.166.94.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47R18h1ctKz4HQd for ; Sun, 1 Dec 2019 21:09:39 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from sdf.org (IDENT:bennett@otaku.sdf.org [205.166.94.8]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id xB1L9Rbj029041 (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits) verified NO); Sun, 1 Dec 2019 21:09:28 GMT Received: (from bennett@localhost) by sdf.org (8.15.2/8.12.8/Submit) id xB1L9Q7d014429; Sun, 1 Dec 2019 15:09:27 -0600 (CST) From: Scott Bennett Message-Id: <201912012109.xB1L9Q7d014429@sdf.org> Date: Sun, 01 Dec 2019 15:09:26 -0600 To: tomek@cedro.info Subject: Re: pendrive clone impossible ? Cc: gljennjohn@gmail.com, freebsd@edvax.de, freebsd-questions@freebsd.org, anatoly@kazanfieldhockey.ru References: <201912010728.xB17SQkC019214@sdf.org> In-Reply-To: User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47R18h1ctKz4HQd X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of bennett@sdf.org has no SPF policy when checking 205.166.94.20) smtp.mailfrom=bennett@sdf.org X-Spamd-Result: default: False [-0.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.966,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; IP_SCORE(-0.32)[ip: (-1.00), ipnet: 205.166.94.0/24(-0.50), asn: 14361(-0.03), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_LONG(-0.96)[-0.957,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[20.94.166.205.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; DMARC_NA(0.00)[sdf.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:14361, ipnet:205.166.94.0/24, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Dec 2019 21:09:41 -0000 Tomasz CEDRO wrote: [freebsd-usb@ trimmed from addresses *again* --SB] > On Sun, Dec 1, 2019 at 8:28 AM Scott Bennett wrote: > > >Does GEOM in any way prevents me from using disk that has corrupt MBR? > > Yes, most likely. ^^^^ ^^^^^^ > > > > >Why I cannot write a MBR from a file but I can from a md0? > > >Any hints welcome :-) > > See the last line of your messages below. > > (..) > > >ugen0.8: at usbus0 > > >umass1 on uhub0 > > >umass1: on usbus0 > > >umass1: SCSI over Bulk-Only; quirks = 0x8100 > > >umass1:3:1: Attached to scbus3 > > >da1 at umass-sim1 bus 1 scbus3 target 0 lun 0 > > >da1: Removable Direct Access SPC-4 SCSI device > > >da1: Serial Number BLAHBLAH > > >da1: 400.000MB/s transfers > > >da1: 118272MB (242221056 512 byte sectors) > > >da1: quirks=0x2 > > >GEOM_PART: integrity check failed (da1, MBR) > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There is your hint, courtesy of > > ||||||||| |||||||||||||||||||||| FreeBSD's GEOM_PART kernel class. > > Hello Scott and thank you for your valuable input. If you are sure > that this is NOT a problem of a pendrive or anything USB related with > that particular pendrive (i.e. some quirk required for valid > operations), and you ARE sure that this is a matter of GEOM, then: Please review wording above. For anyone to make the claim that you ask here without his having physical access to the hardware, or even with access, would be absurd. Having said that, I would just comment that small flash drives are the most frequently fallible/defective devices I have yet encountered. I have used them with no problem, then unplugged them, and the next time I've plugged them in, they caused a stream of error messages on the console until the kernel eventually shut them down and refused to reuse the device name for any purpose at all until after a reboot. Usually, though, they fail mostly silently and become totally unresponsive. Would a flash drive that failed in some way I hadn't seen before be surprising? Not in the least. > > 1. OS / GEOM is hiding things from operator. It does not write bytes > as instructed to fix the disk, instead, it considers disk invalid and > silently discards _only_some_of_the_data_ with no clear error/warning > indication. Unacceptable!!! Well, let's see here. You ignored the error message the system did give you, but now you want more and different messages? The status of the device after the kernel has rejected its MBR is not something I could say with certainty. > > 2. OS / GEOM lies to operator. It does return a SUCCESS code while > _some_ data goes to /dev/null. Unacceptale!!! > > 3. If disk is _considered_ broken then access should be _fully_ > blocked. But how am I supposed to fix it when OS silently blocks > essential part of the fix? Who allows writing over a corrupted disk > anyway? > > 4. OS / GEOM is broken and incoherent in this area and proves system > unreliable / not trustworthy. This needs to be fixed please. Will > report a bug. > I would suggest that you at least try what I suggested in my previous message. Do also take a moment to read the descriptions of both sysctl variables, and think about what they imply. That took care of such problems for me. > > That "irrelevant blather" just proves above. OS cannot silently > interfere with operator actions and do whatever it likes. Other people > also noticed this problem. This is not the FreeBSD way, looks more > like Linux way. I've already shown you that it was not silent in rejecting the MBR on the device. > > Now it looks like the factory pre-format pendrive was considered > invalid by GEOM, this is why the initial `dd if=/dev/da0 of=/dev/da1` > copied _almost_ whole data but without MBR. Then the situation Your description does look peculiar, I agree. However, you might also try recording all command input and stderr output using a tool like script(1), so you could show us exactly what happened. Hm. Before rebooting, here is something else to try. After attaching the flash drive, if you get the error message again, try ls -lgFG /dev/da* gpart show da0 da1 gpart show -l da0 da1 and post the output from them in your next report here. I am curious to see which device names for slices and partitions, if any, the system may create after it rejects a MBR, as well as what gpart(8) gets from it. If you try setting the two sysctl variables as suggested, these commands' output would also be interesting to see in that situation. > escalated - MBR dumped to a file could not be written to a target > drive, but it could be written from a md0 device, and all sorts of > black magic. It THIS IS UNACCEPTABLE!!! Again, another post hoc description of events, rather than evidence we can look at. > > When I DD something from IF then it must get untouched into OF, unless > missing privilege or hardware failure error is clearly reported > preventing further actions. > > Damn, the DD is the simplest thing in Unix. > > Either I get the command executed exactly as instructed, or not > executed at all. The return code is here to say what happened. Error > is here to show me something is wrong. As simple as that. > > Now it looks like the disk is okay, just GEOM did some interpretation, > knew better what I want to, and did, without telling me that. > > Imagine there were some really sensitive backups on the drive and > system decided only to copy selected parts of them with no error. Not > a problem for you? > Please use script(1) to document what happens with the sysctl variables set as suggested by the man page, and then post the results to the list. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************