From owner-freebsd-current@FreeBSD.ORG Tue Oct 8 22:14:37 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D6E30D1C; Tue, 8 Oct 2013 22:14:37 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9DD582669; Tue, 8 Oct 2013 22:14:37 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.17]) by ltcfislmsgpa04.fnfis.com (8.14.5/8.14.5) with ESMTP id r98MEaSg030779 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 8 Oct 2013 17:14:36 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.103]) by LTCFISWMSGHT06.FNFIS.com ([10.132.206.17]) with mapi id 14.02.0309.002; Tue, 8 Oct 2013 17:14:35 -0500 From: "Teske, Devin" To: Nathan Whitehorn Subject: Re: [CFT] Patch to bsdinstall to support root-on-ZFS and GELI Thread-Topic: [CFT] Patch to bsdinstall to support root-on-ZFS and GELI Thread-Index: AQHOxHPFPJPIP8ZCyUiSsjBzr2++AA== Date: Tue, 8 Oct 2013 22:14:35 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D720FC46904@LTCFISWMSGMB21.FNFIS.com> References: <52546844.2010608@freebsd.org> <5254774C.8030204@pix.net> <525478EA.8080207@freebsd.org> In-Reply-To: <525478EA.8080207@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.121] Content-Type: text/plain; charset="us-ascii" Content-ID: <8DD5143031AEDE4E93C15DC8D1DA17D9@fisglobal.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-10-08_08:2013-10-08,2013-10-08,1970-01-01 signatures=0 Cc: "" , "Teske, Devin" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Oct 2013 22:14:38 -0000 On Oct 8, 2013, at 2:28 PM, Nathan Whitehorn wrote: > On 10/08/13 23:21, Kurt Lidl wrote: >>=20 >> On 10/8/2013, Nathan Whitehorn wrote: >>> On 10/07/13 21:59, Allan Jude wrote: >>>> Devin Teske and I have been working on a big patch to bsdinstall to >>>> implement installing on a ZFS pool. It supports both GPT and MBR, >>>> the 4k >>>> sector gnop trick, and optional GELI encryption. We would like to >>>> commit >>>> this in time for 10.0-BETA1 so it needs some testing to work out any >>>> obvious bugs before we send it off to re@ to get it committed. >>>>=20 >>>> It includes a single configuration menu that allows you to select >>>> all of >>>> the required details, including which drives to use (gets details from >>>> camcontrol, also includes an inspection utility that presents the >>>> detailed output of camcontrol inquiry/identify, and gpart show), what >>>> ZFS RAID level to use (taking in to consideration the selected >>>> number of >>>> drives), GPT/mbr, 4k YES/no, GELI yes/NO, pool name, etc. >>>>=20 >>>>=20 >>>> Additional, it includes some other changes to bsdinstall: >>>> 1. Change the default to the 'non-standard keyboard mapping' prompt >>>> to no >>>> 2. Replace the 3 separate dialogs to configure an ipv4 address with >>>> just 1 >>>> 3. Remove the dialog asking if you wish to enable crash dumps, this >>>> feature has been combined into the regular 'services to enable' dialog >>>> and enabled by default >>>>=20 >>>>=20 >>>> You can browse the patches here: >>>> http://druidbsd.cvs.sf.net/viewvc/druidbsd/bsdinstall_zfs/ >>>>=20 >>>> I've built a bootonly.iso (10.0-ALPHA4) to make testing easier, >>>> available compressed (48 MB) or uncompressed (211 MB): >>>>=20 >>>> http://www.allanjude.com/bsd/zfsbootonly_2013-10-06.iso.xz >>>>=20 >>>> http://www.allanjude.com/bsd/zfsbootonly_2013-10-06.iso >>>>=20 >>>>=20 >>>> We look forward to your feedback >>>>=20 >>>=20 >>> Thanks for doing this! I had a few comments: >>> 1. ZFS is not bootable on all architectures. Could you adjust that menu >>> item to only display for i386, amd64, and (I think?) sparc64. Use uname >>> -m, not -p, for this. >>> 1a. The script is broken on sparc64 in any case, which uses VTOC8 >>> instead of GPT. >>> 2. Why are you using camcontrol? That is guaranteed not to work on >>> non-CAM systems. You should use the GEOM ident string if you need an ID. >>> 3. Any plans to integrate this into the regular partition editor? ZFS >>> support is important enough that I will definitely not get in the way, >>> even as a bolt-on, but it would be a shame for it to stay that way. The >>> editor is also designed for ZFS to be added. >>> 4. What is this gnop stuff for? >>> 5. I think some substantial part of the MBR code will blow up if you are >>> reinitalizing a previously formatted disk (the bsdlabel will be retasted >>> and come back from the dead). >>> -Nathan >>=20 >> I wrote some support for adding ZFS to the partition editor a couple of >> months ago, around the time that 9.2 was branched. One of the biggest >> things that I have not done is integrate setting up a zfs mirror between >> any disks before creating the zpool. Nor did I do the gnop hack to >> create 4K disk blocks before creating the pool. >>=20 >> I more or less changed the partedit program to take an argument when >> invoked with "ufs" (same as current behavior) or "zfs", which knows >> about zfs setup stuff. The hookup to the actual scripts was, shall >> we say, much less intrusive than what has been published here. >>=20 >> I guess it's time to dig out the patches and make them available to >> others. >=20 > That would be great! The original idea was to have a "zfs_ops.c" to go > with "gpart_ops.c" and have some fields in the disk item struct that > said which to use. Not sure if that's ultimately workable, though. I can > hack on it once the patches are there in any case. One of the options presented was to integrate ZFS knowledge into the partedit module. However, another approach that is not being spoken about is in the other direction: rewriting partedit to be in sh(1). What I'd like to see is the --treeview widget of dialog(1) "fixed" before such a transition is made, so the former option may be the best bet for now. For anyone that hasn't run the treeview or treeview2 samples in head/contrib/dialog/samples ... it's not anything we'd want to use to replace dialog(3) with. I've seen much nicer treeview implementations than what is in [c]dialog (currently we're using dialog-1.2-20130925 and the treeview for some reason incorporates a radio-button column which we do _not_ want for a disk partition layout menu). Of course, right now you're all looking at me like I'm crazy. Like a fox. There are real tangible benefits to rewriting the C code of partedit into shell syntax. The value-add is not just perceived. Some of the values obtained are: + Conflated namespace with singular API rather than having to learn how to script each individual blocking-utility which implements a unique method for scripting its input. + Top-down integration of new functionality. We're integrating ZFS right now and that's great that we have a fully-functional libzfs et. al. for doi= ng this stuff in C, but usually the command-line utility functionality precedes programatic functionality; it is in this spirit that we can integrate new functionalities into the installer purely from a base utility standpoint (i= f a utility exists to do something, we can integrate support for it rather than [potentially] having to wait until a C library surfaces). + Easier patching for custom deployments. The more code we write in C the more code has to be custom-built for in-house deployments (versus shell in-which you can patch-and-deploy). + More robust data. While it's true that most data (such as hardware descriptions of disk devices) is accessible from C libraries, we stand to gain more robust access to data if we are forced to get the information from a userland tool. Why? because then we make sure that the data is in some-form available from some tool (and not *only* available from a library requiring you to write a C program to get at that data). In other words, the shell approach promises to expose more central methods of accessing informative data *while* at the same time making it easier to bring that data into the installer. I could go on, but I digress. I think I'm making good arguments. I think also we should be mindful that arguments such as: "What if dialog(1) goes away?" (A: dialog(3) and dialog(1) have always been provided by the same vendor -- for multiple distributions from at least two generations of dialog functionality -- regardless of C-based or utility-based) and "But shell is nasty; slow; and not as powerful as C" (it depends in what context; the first is rhetoric, the second is only true for poor implement- ations, and the third may be true in some contexts, but I consider the answer to "how maintainable is it" to be a factor in the "power" of a language, so don't necessarily consider C to be more powerful than shell as the latter is as-or-more maintainable with fewer LoC and a higher return on investment; see previous [above] arguments). And to that, I say... bsdconfig(8) is 100% shell. Run it; Inspect it; .... Love it? --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.