Date: Tue, 4 Oct 2016 02:31:48 +1100 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Warren Block <wblock@wonkity.com> Cc: David Christensen <dpchrist@holgerdanske.com>, Perry Hutchison <perryh@pluto.rain.com>, freebsd-questions@freebsd.org Subject: Re: FreeBSD-10.3-RELEASE-i386-memstick.img installer changes contents of USB flash drive Message-ID: <20161004010853.E6806@sola.nimnet.asn.au> In-Reply-To: <alpine.BSF.2.20.1610011120520.32958@wonkity.com> References: <mailman.4241.1475049553.1479.freebsd-questions@freebsd.org> <20160929014801.W6806@sola.nimnet.asn.au> <alpine.BSF.2.20.1609290658280.7457@wonkity.com> <20161001235138.N6806@sola.nimnet.asn.au> <alpine.BSF.2.20.1610011120520.32958@wonkity.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 1 Oct 2016 11:39:35 -0600, Warren Block wrote: > On Sun, 2 Oct 2016, Ian Smith wrote: > > > > A secondary copy of the partition table is saved at the end of a > > > GPT-partitioned drive. This is a problem for GPT when a binary image is > > > written to a device that is not exactly the same size as the image. That > > > is pretty much always the case with installer images. > > > > Indeed. MBR/BSD images don't have this problem because all metadata is > > at the beginning of and within the image in the case of UFS superblocks. > > So an image can be copied to a device like /dev/da0, in the case of the > > hitherto dedicated drive /dev/da0a, or as easily to a slice like da0s1 > > where it becomes da0s1a, as long as there's enough room on the memstick. > > > > A GPT image seems more of a whole-disk image rather than (what equally > > can serve as) a single partition image. It must have a primary GPT (all > > 34 sectors?) but do you know if these images already contain the full 34 > > sector secondary GPT? i.e, I don't see why only the last sector of the > > copied image file was changed, with no update to any of the prior 33? > > Most of those 34 blocks (not a fixed size, by the way) contain partition > information. But there are global flags and things like the PMBR which could > change. Without knowing exactly what that byte does, it's a guess. Ok. I've been using the Wikipedia page as reference, as I tend to find diagrams more useful: https://en.wikipedia.org/wiki/GUID_Partition_Table I find it slightly weird that the secondary GPT (there) uses LBA-33 for the first four partitions; if it was my design (cough) I'd have reversed that also, and used LBA-2 for the first four, etc. But then, I've never been enamoured of this (problematic) end-of-physical-disk positioning. > Note that we don't know when that change is happening. It could be the > machine checking the secondary GPT on the memory stick that is helpfully and > silently "fixing" something when it recognizes the stick, before it is read. As David mentioned, doing so silently would be less than helpful (POLA!) > It could be malicious or buggy firmware in any of several places, including > the UEFI or BIOS or the memory stick micontroller itself. The point is that > we don't know that it is the FreeBSD installer making that change. Right, so I guess we need more people to test this on their machines, but I'm not in a position to do so lately .. maybe David's PR will help. And Perry's issue is also outstanding, and rather inexplicable so far. > I remember a few years back that someone helped me understand what was going > on, but forget the details (and, unfortunately, who it was). Many of us could relate to that :) "I thought I met a man .. who said he knew a man .. who knew what was going on .." David Crosby - IIRC! > > Personally I'm wanting to get to a memstick with 2 or 3 bootable slices > > for installers (amd64 and i386) - as least the amd64 one from dvd1 with > > packages - plus a mountable r/w data partition. Using only native tools > > currently implies using boot0, thus MBR/BSD schema. Converting a new > > GPT-scheme memstick image to ONE slice on my new 16GB memstick is what I > > need to figure out, possibly by means of using a tar pipeline similar to > > dvd1_to_memstick.sh. I hope to get time to look at 11.0 make release > > memstick scripts, but I'd appreciate any clues / ideas meanwhile .. > > I'd think you could just dd(1) the contents of the single installer disk > partition into a big-enough MBR partition. Repeat for each image, then use > boot0cfg. Sure, for old-style raw BSD images, but that won't work for images with: 1) PMBR, 2) primary GPT, 3) efifat partition, 4) gptboot partition then 5) the UFS partition - which is all we want to copy to a bootable BSD slice - then 6) 1M(??) swap. We need to use /boot/boot bootblocks for each slice; still, it may be easy to locate and dd just that partition out of the mdconfig'd image - or mount it and copy; some tests needed. > It really is a shame we still require separate installers for amd64 and i386, > a single installer can install either. It's got a lot simpler on 11 src/release, using mkimg(1) rather than scripting gpart steps as before, but both memstick and iso still differ between i385 and amd64. Perhaps the EFI issues are the main difference? root@x200:/usr/head/src/release # diff -u i386/make-memstick.sh amd64/make-memstick.sh --- i386/make-memstick.sh 2016-05-22 06:27:55.712694000 +1000 +++ amd64/make-memstick.sh 2016-05-22 06:27:55.713721000 +1000 @@ -7,7 +7,7 @@ # # Usage: make-memstick.sh <directory tree> <image filename> # -# $FreeBSD: head/release/i386/make-memstick.sh 293188 2016-01-05 03:20:45Z gjb $ +# $FreeBSD: head/release/amd64/make-memstick.sh 293188 2016-01-05 03:20:45Z gjb $ [.. clipped and reformatted for clarity on the list ..] -mkimg -s gpt -b ${1}/boot/pmbr \ -p freebsd-boot:=${1}/boot/gptboot -p freebsd-ufs:=${2}.part \ -p freebsd-swap::1M -o ${2} +mkimg -s gpt -b ${1}/boot/pmbr -p efi:=${1}/boot/boot1.efifat \ -p freebsd-boot:=${1}/boot/gptboot -p freebsd-ufs:=${2}.part \ -p freebsd-swap::1M -o ${2} cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20161004010853.E6806>