Date: Fri, 5 Mar 2010 18:59:58 +0000 From: xorquewasp@googlemail.com To: jhell <jhell@DataIX.net> Cc: Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= <des@des.no>, freebsd-hackers@freebsd.org Subject: Re: package building failure irritation Message-ID: <20100305185958.GB82765@logik.internal.network> In-Reply-To: <alpine.BSF.2.00.1003050917290.7404@qvfongpu.qngnvk.ybpny> References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> <20100226222113.GA14592@logik.internal.network> <4B884D48.90509@FreeBSD.org> <20100227093409.GA40858@logik.internal.network> <864ol0w4g5.fsf@ds4.des.no> <20100304175819.GC31036@logik.internal.network> <867hpr56ek.fsf@ds4.des.no> <alpine.BSF.2.00.1003050917290.7404@qvfongpu.qngnvk.ybpny>
next in thread | previous in thread | raw e-mail | index | archive | help
'Lo, On 2010-03-05 09:47:11, jhell wrote: > Adding on to this. There were reports in various cases dating back to ~1 > year with bad results, possible data loss, hard and soft dead locks when > nullfs was used with ZFS. nullfs at one point that I do remember was not > recommended to be used at all with ZFS and there exist quite a bit of > other functionality in ZFS "cloning, snapshots, mountpoint=" etc... that > serve well enough to not use nullfs at all. Except, as far as I can see (not far), there's no way to emulate one directory/fs being mounted in multiple places. I'll come to "why" in a second... > Surely in the case above you are talking about packages and in which you > really should not need to mount this multiple times as a writable FS, > correct me if you feel that it does and lets see why (please provide your > process if you do). Basically, what I'm doing is trying to share a writable distfile directory across multiple package-building jails as a sort of cache. I don't have unlimited bandwidth usage per-month and would like to avoid multiple jails downloading 150mb distfiles (I've already exceeded monthly bandwidth limits due to this exact problem). I'd also like to avoid having multiple copies of distfiles laying around (as the number of jails increases, the space required becomes not insignificant). A possible solution to the above might be to use "make fetch-recursive" on the "host" and then just mount the distfile directory read-only inside the actual build jail to eliminate one of the read/write mounts. Of course, what distfiles are actually required might change based on the settings in make.conf in each jail... I'm also sharing the ports tree read-only across jails to ensure that each jail has the exact same tree. This isn't a problem as far as I know (due to the "read-only" part). It might be the case that the only real problem (the one that causes the package building errors) is the read/write nullfs mount of the /work directory so I'm about to build a test jail now and see if this is so (and I'll use DES' method of setting the ZFS mountpoint for this directory if it is as there's specifically no need for shared state here). > This should suffice mounting a packages type collection in multiple > places: > > # Mount one dataset wherever you need it. > zfs set mountpoint=/path/to/wherever pool/packages > > # Create a snapshot and clone it for further mountpoints. > zfs snapshot pool/packages@20100305 > zfs clone pool/packages@20100305 pool/packages2 > zfs set mountpoint=/path/to/other/dir pool/packages2 > > If none of this would suffice then your package management needs to be > re-thought out. Central FTP, NFS, SMB, RSYNC?. In this setup, jails build packages for other jails that specifically don't have network access (see the example given before about sandboxing a pdf reader): 8.0-amd64-mistrust_pkg mounts /storage/jails/8.0/x86_64/mistrust/pkg at /pkg and writes built packages to it whilst 8.0-amd64-mistrust mounts the same directory read-only at /pkg and, of course, installs packages from it. In this case, the only sane way to do things seems to be to mount a read-only view of a directory of packages built by another jail with nullfs. I don't particularly want to be cloning and managing snapshots - I'd be exchanging the simplicity of a direct solution (a direct, read-only view of the current state of a given package directory) with a solution where I have to endlessly shuffle and clone snapshots. > You have plenty of options available but the only one that will be > suitable to you & your process will be the one that is planned for > accordingly and thoroughly. I'm surprised that something as simple as the above requires this much thought to be honest. xw
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100305185958.GB82765>