Date: Mon, 17 Mar 1997 19:25:36 -0700 (MST) From: Terry Lambert <terry@lambert.org> To: msmith@atrad.adelaide.edu.au (Michael Smith) Cc: terry@lambert.org, msmith@atrad.adelaide.edu.au, bde@zeta.org.au, dgy@rtd.com, hackers@FreeBSD.org, helbig@MX.BA-Stuttgart.De Subject: Re: wd driver questions Message-ID: <199703180225.TAA08818@phaeton.artisoft.com> In-Reply-To: <199703180112.LAA12765@genesis.atrad.adelaide.edu.au> from "Michael Smith" at Mar 18, 97 11:42:28 am
next in thread | previous in thread | raw e-mail | index | archive | help
> > Bullshit. It is possible to get it right. You may have to rewrite a > > scratch area of a disk in order to do it, but it's possible. Windows95 > > does it just fine. > > Ok, so I give you a pathalogical case. I can think of several right > off the top of my head; most involve flash memory or read-only media > of some sort. > > Of course, you have no way of knowing that the area of the disk that > you've elected to use is actually "scratch", do you? Remember all > those nasty comments about MS operating systems eating peoples' disks? The "cannonically correct" way to do this is to get the protected mode driver up in parallel with a VM86()-based INT-13 using driver, and if you can't tell two media apart, write on one of them with one driver and notice the change with the other. You either: 1) Save the previous contents of the byte you write 2) Restore the byte you wrote from the absolutely identical media Either way, it is fully recoverable, especially if you choose an LBA in the partition table of one of the drives, or if you have FreeBSD recognized to partition types (A5 and one other) and switch the FreeBSD partition tag between the two possible (but otherwise equivalent) values. If they are read-only media, *it doesn't matter*. For "less cannonically correct" code that can't call VM86(), you bail: ask the user, and cache the answer. This is less horrific than asking every user during the install, defaulting to the "wrong" values for a shared disk install (the most typical install), and failing to run on all modern Dell, Gateway, etc. hardware without the user futzing around in fdisk. > Given unlimited resources and patience, I'm sure you could do that. With > the much scantier resources actually available to the bootloader, things > aren't so easy. Linux does it. Isn't FreeBSD as good as Linux? Which particular resource do they have that FreeBSD doesn't? I feel low trotting out the old "Linux" card, but you leave me little choice in this dialectic. Hopefully by "unlimited resources" you were referring to MS? > You are making the (possibly incorrect) assumption that the bootloader > has the 'correct' geometry in the first place. This is only valid if > the bootloader obtained the correct geometry, and has not been misled or > made a mistake. It could do this if it didn't come off the disk in > question, or if the BIOS in the system is behaving oddly, or if it > was loaded by something other than a vanilla MBR. If the bootloader was misled, DOS was identically mislead, and since all we really care about here is interoperability with DOS installed on the same drive (or another OS that must also care about interoperability with DOS installed on the same drive), it is *correct* to allow ourselves to be "misled". In the "mistake" case: bad code is bad code and all known bugs should be fixed instead of ignored, especially when it comes to something as critical as people not being able to install your product. > > If you have read the VFAT32 specification that came on the OEMSR2 > > CDROM from Microsoft, you know that the code looks at the partition > > data of the disk in order to determine the geometry (just like the > > NCR BIOS does in order to make other SCSI controller geometries OK > > for the NCR controller). > > I have little or no time for reading Microsoft's confusing and misleading > documentation; I have far too much work creating my own, thanks. > > Oddly enough, not everybody subscribes to, or _wants_ to subscribe to, > MS' developer programs. Personally, I get by quite nicely avoiding > subsdising immoral corporates wherever possible, and MS are right up > there with Shell and McD's. The problems with the boot are MS OS boot interoperability problems, nothing more. If you have an issue about dealing with interoperability problems by examining the software with which you are attempting to interoperate, well, all I can say is that you are unlikely to ever solve your problem. > Reading the partition data on the disk also requires that you are using > an MBR in the first place; another poor assumption. No, it's not. You should *always* have an MBR (or whatever passes for a native MBR on non-x86 computers -- on PPC's which comply with either PReP or CHRP, it's a DOS MBR. On DEC Alpha's, it's a DOS MBR). This is common sense, since it reduces the number of cases you must consider when you are trying to recognize the thing. If you are guaranteed that the first stage boot won't be a BSD second stage boot (you can do this, because we have OS-BS and a number of other programs at our disposal, something which occured *after* Bill Jolitz implemented this case for 386BSD installed on a system on which DOS had not been installed), you can make all sorts of nice simplifying assumptions. > I am fully aware that the bootblocks have access to the BIOS information. > I am also keen to make this information, as well as a lot more, available > to the kernel in a useful and disposable format; I just haven't > received much in the way of support or time. It's there already. It's in the same variables. It's utility is being declined because of the identity problems (already discussed). > > o Multiply the C/H/S geometry of the INT 13 ID's in question by > > the C/H/S limit values to get the absolute drive size. Compare > > the size against the IDENTIFY size. This will establish disk > > identity in the vast majority of cases. > > Huh? It is almost certain that, in the majority of cases, the C/H/S > size will _not_ match the drive's total reported size. You would have > to apply a 'nearest fit' heuristic with some intelligence about > what is 'near'. Not impossible, no, but not terribly simple. Size-ordered BSD device list, bsearch for BIOS sizes. *Very* simple. > > o Modify the second stage boot to fill in the correct LBA address > > in the DOS partition table for a given C/H/S value at boot time; > > have BSD *always* use the LBA values in protected mode and > > *always* use the C/H/S values in the INT 13 using boot code. > > This should cover all other cases except where the drive is > > identical. > > This would be nice. It does, however, require that the second-stage > boot be able to write to the disk, which is not always practical (and > on many systems will drop the user into a large flashing red "virus > alert" screen). Only write it if the values are wrong. After you write it once, they aren't wrong. In the "Virus alert" case (few machines offer this at the INT 13 level anyway), instruct the user to override. We must already be instructing the user to do this to install our partition table entry anyway, right? One partition table write can not be distinguished from another. A write of that disk address range is a write of that disk address range. > > > The correspondence between FreeBSD device and BIOS device is only > > > significantly relevant with a mixture of SCSI and IDE devices, or > > > multiple SCSI devices on multiple controllers. > > > > It's relevant when there are two or more devices in the system which > > may be identical except for the INT 13 drive ID. > > The only situation in which it is justifiable to be concerned about which > physical drive matches which BIOS drive ID is when attempting to determine > from the BIOS disk number of the root device from the bootloader which > disk contains the desired root filesystem. In most cases, this is trivial > and the whole argument above is moot. However, it is desirable for things > to work in _nontrivial_ cases, so we are already talking about > border conditions. Yes. > > What's wrong with Erich's code to do this? > > The fact that I have yet to obtain a version of GRUB that can be built > on FreeBSD, or that comes with anything approaching the sort of basic > functionality I'm looking for? I'd suggest contacting Erich about this. Are you using modern tools? Erich has typically been very responsive of all emails on the subject of GRUB, at least to my reckoning. > GRUB is a nice idea, but too limited. I would very much like to work > with it, but if I can't build it, I can't tinker. In particular, the > hardcoded blocklist and "stage 1.5" concepts make me want to puke. > It is also salient to note that GRUB has not been updated for over > 5 months, making me fairly certain that it's a dead item. Or perfected. ;-). Not that I believe that, in case the wink made it unclear... > My current 'favorite' is the NetBSD libsa, which has all of the bits > that I need, modulo perhaps some more network drivers, however it doesn't > appear to have all of the "wonderful" ELF stuff that you keep raving > about. I've never raved about ELF in NetBSD... not even in their boot code. > > I don't see why the stupid one could not be the same as the smart one, > > as long as you were willing to establish PM mappings for it. > > *shrug* The idea behind having two was simply that the 'stupid' one > need only be able to assemble a kernel from a specified list of > components, while the 'smart' one needs to run after the kernel has > started to add extra components. Actually, you want to be able to load all segments with a certain set of tag bits from the kernel image file, which is a single ELF file. This should be a lot easier to do, actually. If you don't care about non-kernel files in the boot stage, then you are done. Othewise, you must provide "vnop" level correspondance between the interfaces, and in the kernel case, provide a "stupid" INT 13 "vnop" layer. The lack of a coherent kernel level vnode-as-descriptor based file I/O subsystem is one of the things that is wrong in the FreeBSD kernel. The vn_* routines are seriously incoherent compared to, for instance, the AIX kernel file I/O interface. > > As for > > segment loading, the most interesting method would be to put modules in > > seperate ELF segments in the kernel image itself, jamming them in or > > yanking them out at will. The segment coloring is only useful in this > > case if your defautl drivers for the console, etc. are colored differently > > than the rest of the kernel to allow them to be distinguished (and dumped > > at will for replacement). Given this, then a "boot" coloring (see the > > Microsoft Developer Network document on the Portable Executable format > > for ELF segment coloring) could get loaded leaving the rest for later. > > It is pointless to make sideways references like this to documents > that are effectively inaccessible. If you have a URL, cite it. If > the document is not freely available, then either paraphrase enough of > it to convey its details, or avoid referencing it as though I should > have read it. The connotation is annoying at best, and I find it > quite insulting. Sorry; I thought you would have gone to AltaVista and typed a search key of "+portable +executable +format" before stating it was inaccessable. Here is the URL. http://www.microsoft.com/win32dev/base/pefile.htm > Look, we're talking about doing something new here. If segment > colouring is useful, and I suspect that John could make it so, then > there's no reason not to do it. We still need an in-kernel (or > attached-to-kernel) ELF linker that can do this. Segment coloring refers to attribute bits for VM handling based on which ELF segment the page in question came from: 0x00000020 Code section 0x00000040 Initialized data section 0x00000080 Uninitialized data section 0x04000000 Section cannot be cached 0x08000000 Section is not pageable 0x10000000 Section is shared 0x20000000 Executable section 0x40000000 Readable section 0x80000000 Writable section (This is not comprehensive: I have additional bit definitions that I have reverse engineered from using the "MODBIN" utility to set the executable attributes, then dumped them out with my own programs; a trivial exercise; for instance "PRELOAD", which is not honored by Windows95 -- test: insert VC++ 4.2 CD, let it pop the install screen, reove the CD, and move the mouse over the install screen and watch Win95 segfault). In addition, the segment identifier, which in MS binaries is predefined in the SDK and DDK as #pragma manifests interpreted by the compiler as attribution, and the .DEF file, can be defined by the linker using the "/DEF" directive. Segments are named using a "section header", the first 8 bytes of which are a null terminated string (0-7 characters) containing the segment from the .DEF file and/or #pragma code_seg(): #pragma code_seg("_LTEXT", "LCODE") ...identifier is "_LTEXT" ...class is "LCODE", which implies... ...attribute "code section, section is not pageable, executable section" The loader would load segments of a given attribute and a given set of identifiers. > All I'm suggesting is that _I_don't_know_enough_about_it_ to do it, and > I am asking that you consider putting some code down that does. There is sample code available for download at the URL above. If you need more, I've duplicated this effort to be able to load symbol tables from Microsoft Debug format and from SoftICE files into a non-statistical profiler for VXD's, when porting the Heidemann framework to Windows95, so I suppose I could give you some similar samples, though my header file for the reverse engineer was not nearly as complete as the ones on the URL above. > > If you read the Schulman book on Win95, this is actually how Windows95 > > loads its VXD's and other pieces... only we don't have an IO.SYS that > > understands our file system (well, we don't have a publically available > > one, anyway). > > As above, please don't suggest that I have time to lie about in > traction reading expensive books that my employer doesn't want on > their shelves. On the sort of peanut salary I'm on, a $90.00 book > isn't something I'm going to spring for just to read that MS use an > in-kernel linker. Big yay. > > Bruce has already observed that the stage-2 boot can happily read UFS > filesystems, so I don't know what your problem is there. The Schulman book is a good documentation of the process of bringing up a protected mode OS from a real mode boot loader without losing access or validity of real mode data. The problem is in conversion from real-mode (INT-13/INT-21 based) disk drivers over to protected mode disk drivers. I was simply giving an example of how it had been done. Don't you have a technical library in your corner of the world? In any case, I doubt we have to go so far as to convert open file pointers from TSR's loaded by autoexec.bat... The main point is that the real mode code and the protected mode code for the VXD (PE format image) loader are the same, so there's precedent that it's possible to implement without a "smart" vs. "stupid" loader dichotomy. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199703180225.TAA08818>