From owner-freebsd-ports@freebsd.org Wed May 29 14:32:29 2019 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE9E215C4817; Wed, 29 May 2019 14:32:28 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 819306F2CD; Wed, 29 May 2019 14:32:28 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-402.fritz.box (p200300CD5F0B620098C1EFFA06128F6D.dip0.t-ipconnect.de [IPv6:2003:cd:5f0b:6200:98c1:effa:612:8f6d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "st_esser@t-online.de", Issuer "WISeKey CertifyID Standard Services CA 2" (verified OK)) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 8BC321E01A; Wed, 29 May 2019 14:32:27 +0000 (UTC) (envelope-from se@freebsd.org) Subject: Re: GSoC: Separation of Ports Build Process from Local Installation To: Theron , soc-status@freebsd.org Cc: Bakul Shah , freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org References: <5cdb1c0b-a2dd-c754-daa3-187330ad9ad6@gmail.com> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= mQENBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAG0J1N0ZWZhbiBFw59lciAoRnJlZUJTRCkgPHNlQGZyZWVic2Qub3JnPokBVAQTAQoAPgIb AwULCQgHAwUVCgkICwUWAwIBAAIeAQIXgBYhBKNx6mWcC+zIK3FTE0frte9a/fVEBQJa8u+q BQkLJQETAAoJEEfrte9a/fVEOeMH/icmdK1eZQvB3U8quJo9VMaZsaTuCMbUE4NThyfsIvIm MCd+rb/yULmMYwqNfjyKB1x4ikR4x+94l+yJoz7K0Usks+eNKDmMGJM6pWWssTigaJubFdVd hVVC+C1QJi7JshYSib08uONoPmO4lv5Az0TDYGtsMzsES2sIlc62c9go5WPGYhQFRbX3Lk6y V6m8OHh+G9XGSj3oPO4UteRwu+SzTdOLunZBWG1wu34+IeZm663D+2gOppQLWpLa2qaTerqw THu377ayZ2B2LPJ5JkvkZeHYPkwDQ+b5PGn0UhfkxPnDVYki5F7qKxvQ5uq1/q9YaCX7mmOl H2yO7tgVsrW5AQ0EVXGJEgEIALEj9qCXMZVucjpcd3QxM/TlUr98m5viEd1z4tCnPUyRWcIC EVtj2h5xMH+2iB0q1+KWhq+NsWtvScmEmfHnsr7dJ1K677OdpDhKVaJk61eeRulFY1R4yb6C 1MMxK+WgYB+vvpG0UeyR0M4uBewcPvRsq4yGUHFQKtLAbMdoPTSryJA+ElnmK1vdY+rPcHgi OIMBZM7ahsPXC0C9K4e5SP9clGyIoMpbfHXdx9q+Rp3zVtlbhyk3BS/xccu/+9pk9ICXL6GR js2sNnJ0wxdU1DsAlC59a5MnSruwiZFwRnkQhr3x6wk97Lg7sLS9jjTnCN7LGlVmSmpOEMy6 uq1AWfUAEQEAAYkBPAQYAQoAJgIbDBYhBKNx6mWcC+zIK3FTE0frte9a/fVEBQJa8u+rBQkL JQEZAAoJEEfrte9a/fVEuesH/2DNxGWnHvWwMyiyhlQtafvDKwEn/wAgR8gHJFodB7emf8rA TnukH7MVttCoHtjN5lvv9RSBHjNTZls5wR/ANlwdRuPQHd8ZGxLe3S6IuUB3zDSwFltLGurO N2kOMhs5mTGyypSa+uw3rtQbUAVYf1oPbiR4FLtiM8FLyEvE95hX5fPq9Qvx9FmN79kmCIEw jDKPqDaUf/OR2fEF0LSIbXHEk4tNqCEwx5DIJ0fp5/z5UzICUAmwxyRs5O/Hre1jzPsMVyud Ml9t7UTOJGKVWwRory1PMnOFxN+iz5/d4FhYSKXF7kfMiFgol4LuWaxJRwbBrr71VGBrRy2a L1nw6Bc= Message-ID: Date: Wed, 29 May 2019 16:32:22 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: <5cdb1c0b-a2dd-c754-daa3-187330ad9ad6@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 819306F2CD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; TAGGED_RCPT(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2019 14:32:29 -0000 Am 29.05.19 um 00:51 schrieb Theron: > Hello All, > > For Google Summer of Code 2019 I am working on FreeBSD's ports tree makefiles > towards eliminating the dependency of the ports building process on the local > system's installed packages.  Currently this level of separation can only be > accomplished in practice through chroot or Jail.  The project will eliminate > the need for cooperation of the root user since /usr/local will not need to be > touched. > > The major technical obstacle to be overcome is that ports expect to find files > of their dependencies installed in /usr/local.  To support this without > touching that location on the installed system, file accesses will be > redirected to a location controlled by the ports build process through use of > a library to intercept file accesses. > > Once I have that working (well enough to build one port at a time) I will move > on to modify bsd.port.mk itself (and related files) to utilize this mechanism > for virtual installation of port dependencies during builds. > > The full project proposal can be seen at > https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit > . What's wrong with using chroot to provide a clean build environment? That is what synth does, and I have been using my re-implementation of portmaster for this purpose for some time, which uses a chroot jail with read-only null-mounts of all relevant file systems and a clean copy of some files and directories in /etc and /var that can be written without root privileges. The jail is set up in not measurable time (irrelevant compared to the time required to build the port). The only problem with this approach is that it requires extra disk space for the build environment (e.g., the specific C compiler required by some port) plus the work space for the actual port build process. I'm using tmpfs file systems within the jail for the work directory and the copies of parts of /etc and /var that need to be written to. Is there a risk of mis-use of the interception library to attack the system, BTW? [Its use is not restricted to root and it might be used to re-map file system paths for commands that check e.g. policy files to decide whether some operation is authorized ... SUID programs should not be vulnerable to such an attack (since they do not allow the library pre-load required to intercept the file operations), but there might be application programs that are restricted by non-writable files in hard-coded directories that could be subverted this way ... (such a command would be ill-designed, since any user could compile her own interception library, but providing such a library with the system and possibly having hooks for it in libc might simplify such an attack, especially if there is no compiler and easy way to install such a library on a host).] > My goal is that this work can be integrated well enough into /usr/ports/Mk so > that unlike Jail, no set up work should be required for using ports tree to > build a set of installable packages. Yes, this might be beneficial. But there will be huge differences compared to the current build process. And in the end you'll probably have to put the logic used by, e.g., portmaster to track dependencies and determine the availability of up-to-date packages (to use as build dependencies) into the ports system. > Please let me know if you are interested in this project; feedback is > appreciated.  If someone would like to provide ongoing feedback or mentorship > that would be especially helpful.  Bakul Shah is my mentor officially for GSoC > but I would be happy to have additional support from someone who is > experienced with internals of the port infrastructure makefiles. I'd be interested to get further information about your approach and the progress you make and my experience working on a somewhat similar project with portmaster might allow me to answer questions or provide some help ... Regards, STefan