Date: Tue, 18 Mar 2014 09:38:44 +0100 From: Adria Garriga <adria.garriga@gmail.com> To: freebsd-hackers@freebsd.org Subject: GSoC: Deterministic builds Message-ID: <CAH_Fh2gpok_-fKyRmJEKgBwX03e9EnEJSM5cCWuf42z9LCAVcA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello! I'm Adria (rhaps0dy in IRC). I'd love to participate in the GSoC as a FreeBSD student. Here's the project I have in mind, could you please give me some feedback? Deterministic port builds ========================= Scripts to automatically create a jail or bhyve guest where you can build ports deterministically. On any amd64 host, given an updated ports tree, same CFLAGS, and this jail, the result of the build is byte-by-byte equal. Progress towards this has already been made by several free software development teams, most notably the Debian and Bitcoin teams. Debian have several deterministic build scripts and documentation on what hinders the accomplishment of this goal. It's mainly the time, the build path and the locale, so with libfaketime or an appropriate replacement, canonical build paths inside the jail and using the same locale much would be solved. The Bitcoin team has produced Gitian, (https://gitian.org/), a set of scripts that create an Ubuntu virtual machine on qemu and run deterministic builds on it. On the (few) tests I've run, clang{,++} produce deterministic object files. Seeing that this much of work is already done, I can say that this is not enough for two months of intensive work. That's the reason I've come up with these two extensions. Extension one: -------------- Extend the above procedure to more architectures, ideally all architectures supported by FreeBSD (i386, amd64, IA64, sparc64, powerpc and pc98 if I'm not mistaken). This would require more testing work and almost certainly the use of virtual machines instead of cross-compiling environments, further complicated by the fact that I only have access to i386 and amd64 hardware. Extension two: -------------- Create necessary programs for a volunteer-driven compilation farm. This could help compile 'custom' ports for users with slow machines, or redundantly compile and verify the official binary-distributed packages. It would be either p2p or centralised on a server. I'm aware that this is a huge task, too big for a single month (half a year even). The goal of this extension would be to produce software for a low number (at most ten) of hosts assumed to be always up to share compilation resources and redundantly check for differences (draft at http://cf.bagelbox.org/freebsd-farm.png). To further reduce complexity and make it manageable, it will only compile one reasonably big port (for example Firefox) but will be easily extended. I'm specially interested in the second extension, really. The first one is a bit of unpleasant but necessary hard work. What do you think? Thank you very much! Adria
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAH_Fh2gpok_-fKyRmJEKgBwX03e9EnEJSM5cCWuf42z9LCAVcA>