From owner-freebsd-ports@freebsd.org Thu May 20 18:48:49 2021 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9BA8C6463A9 for ; Thu, 20 May 2021 18:48:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmJfm2GHtz3s9Q for ; Thu, 20 May 2021 18:48:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621536526; bh=cKgcyS5AvF3gfVgUyfrHmTJt+Si/AKSq133JIX/7CGt=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=r64uPzbveWQGd6ohhU5u9dDaqO9WkMmEZPcxVTvuXR4FLhKgc9oiwTjdNoVgwJmDIXh8zFmTTF5S7naGbtmovch+D1+bJNgZ4SdUb9pn9NDVp8Amp+jbC+nKjPoWDd4cWKYgxgrpPK0X4Os62havC7utRf1m21ReKSJoC9+M5cFl4oe5gx3eXIjm0jVXYg0GIiDWlF0j+lP4GsPAVF95NjMqudUFS7dNQDDJFNs63q1y0rASQ9aZWScxsLNwKJDKrxjbSVlsl5DZASb4rrlGKhXJFmp5T2iJ4heGyXACjXCtO8sGsbdqB9D3udJPCoOkbDxnQRGbtQz+DMoz5sibEw== X-YMail-OSG: SWfRWRYVM1ljGSY8AwUTRYY9bHEd3pv5kgt2KnhzqLmB1JK7PnDnrfveOMXpaxF qV1mKiisS60V2oEHicGEZplwYT0.wMZkl2UUHQT.e6ry1q7k8NbGEZHWnGzSM3shgEm3f5zxhwS2 QXXIoxxUT_CUBuiarHAfXuY2MujLj3Ug5nKxMvD1D2QfQsjBj2iLyAteAkUvfR_28r4GilTGrET8 vhjfwhUpnhpWIBH3Mhdt1RMTDcuONufazeGnXDbD6cKS8x242kBz0TXRO4jSo3XsZEotS0QvtF1O gC6XtOjgdnKa2.659aCt_VmstmfAuZfDtYkFf9eaQYZXdTg_15CSOsAuxKaqkxeNfgM7q9PIbAGu nKgdBOw82c._KcZeO6B_uMWigHeIztn7cTyAjMptUssLWxmaHs_wVJloyc3rY9.xnxj4NHNB0AB9 O7.2nSfzay6q56uKpq5IiRi7WGjWdLN0ZL2S3V7r4F9JrNu6gYMJEJZgwrpGYDVatGB6i9FBLYrY msm3AbYA.xLlMzItwX_5V_i5UiBbAYWlAKJ9vm2ZO9W75psOFx2Di8tPHye4MZqmFZrMR4rImj1j n0baJP2N7Zs_uUA2t2kLytery0TSrvPr8qIlCdWGwHfUhT_7w2lfXk1LqUv8XtZCSRG_L.wxX8yp zG6ruX_vX5xC7UdYIo.JLTY9QvaMaBoxOAwldJYJZzLxB0wb.0taBFhlhykUCxil4xcbFYrZyPsT PVHxxrzejI1IDOg7LoHTLcQL_dVZWOy1JG1kgGk7kGck.569y3r6YU_a81hmQdW0miqj.9zyAlaj ckxxTdFhNg4IyYjUut9CdF9QIdIgsUOGyj.OaZVzup29TQSir4MYo_A4z1274aciH8uhMYV2RpXl DQi6ahrJapFVND8n127qcDvHkOP_G6Ipmd7hkYYCmTIh_XwK1iPJzHF2Kezx3Y864u5iaLKOP8YG ory02szyZ8HSbqTRf8_b82nKe2CsNMiE4JuDD_k0gGvPuTMqQlCqfP4HxJOLvRNBhqO.NQQ9ZSbK .bJ3rEWk2NxF77cTK0bXuWc7GfpTFXoImim0eQNC2Oni3Jxh122lZ.in8LQEIWRv_utu8LG1L6YO _nBS1GLrWLJEMMxN3iobSx2L8dc_YP.Tzim1UEdq8t5l8sSyXqB91IVP8bjJYHEjfUXOgOXuEYsI 6..Eg_lN0qiaNJeJLqIm6_8QfgJBA5o2jD42Oqvh00ih9eV8BUttI3uxNsdJn4CpVyogF53zM7tz mGDrrStiFbs7cRM2CpeRelDmPkCLZgZXC6wGw9rukjZMuoR6bAZrO3ZDw2OX.jlXbFL7OnNn4FnC y69HccAFMFfvjcww5KVL8AwMnmkPWnG2I3taNhKsNqawZ6PEqojSp3Me4APyMU1xEGG0kqmbkH5B FlNcnrObY8dRg_mWAXVxJR3eSS0X7KNw8HbdmdOsAieDz3Tie8DT1gmEt6aFztrg0A_A779hSXFR cOQwdBnB9zazkL77czEkPZ8.2qLXgQjAjTXOdnzgp0GA3iuO7iUWH1OEM3_XSsBdBnimzeRXiyu8 DB2B57GeOg0u0enWuONVmoi1BdbliHbHwt1z..QjRm5XEfFDz9siApe5MrKTUgSErZqZ.ZRn_rbu aIP891o2xTXZxvuYYJbh4IPrXEqG53Zusa76DYKk2GxccAf6IoPzcsOOpuvMku_Z8ZJffOx15sni NFE5fwOQF1nEwo0tEteVM4YH7o_D01XJDCiKAp0g2pWlzBirLZesOl7WZXMJ7gMYcLfm.0S5bx13 glkSq_yi6RGEduUKrsraSBwJRlKOEMz0QV.NB1CEMEEdUm6o0CO0ai9LIgbgLA_ugCM7YbEK8GA5 ZsIRkwue.RZKDLSPUN1nnhGEpIcoGXd0FdqaejN0orY.WsPwE979hM069aHApGHDlu6pboJ5pcMb ttxgSwG14fZNQe8im1ba0AvFLUvysK1Rgyn.0AbmNdQ4BAjzUegIt7I5geI5HbbSohJzoLDafNYk k4acMfamK3LIJfWSakbq3ivjUKrxBKLqPFfLpfTiXjftApRz8xMNXzqyR2a3X7ZjC8hRTUsbYvCI ASXrCPJaD7d.pSKho4F5CBI0ko5wv7mIsySv3.qsopgX5i6aezJK_yHYYXvFPPD_7DbfFmLVOdzM Ait5uCtpZErlLNta9cshSedR2yz5vpfXlLv_NBzJ8q865XNoBHm8KBS0c5_1yE6LO83cZfaI2tsJ 063NrpSGt4y39rWj48Iu8fbG4JTrmMh5WNydgUuV068PT9QpBKh0EmXrRJ20hfC9fedJxj4nEkoT j.LRCCJ0OXqhPqSRqlqsp0koYTMMsnAey1KwsVIZYAgM.PXbrw89inKFOL5P456JfQuox7RiFWqC APozNUStplzmGOch6vvHJ8mgDehqfViwSNu8gE6a3D8br5R4nc7Pnu.m2TS0ohgS3ecuRps5BNVd oNk1c40h0y.te5kvV1vEncrp_pzXFU0dPp_dPE.83qzQ- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Thu, 20 May 2021 18:48:46 +0000 Received: by kubenode513.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID d5647228ce8b5cbd344ef9a3de1dc99f; Thu, 20 May 2021 18:48:42 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: Large builds with poudriere Date: Thu, 20 May 2021 11:48:40 -0700 References: <6307B6F4-87CB-4C8C-A3E9-11089B13FDAF@yahoo.com> To: ml@netfence.it, FreeBSD ports In-Reply-To: <6307B6F4-87CB-4C8C-A3E9-11089B13FDAF@yahoo.com> Message-Id: <4F3B2B9B-1B55-4500-A93A-2AC765BE4770@yahoo.com> X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FmJfm2GHtz3s9Q X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.83:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.83:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-ports] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2021 18:48:49 -0000 On 2021-May-20, at 11:11, Mark Millard wrote: > Andrea Venturoli ml at netfence.it wrote on > Thu May 20 14:54:50 UTC 2021 : >=20 >> On 5/20/21 4:11 PM, Alexis Praga wrote: >>>=20 >>> Thanks for the suggestions. >>>=20 >>>> At least, if your machine is powerful enough, you can edit your >>>> /usr/local/etc/poudriere.conf and list these big ports in >>>> ALLOW_MAKE_JOBS_PACKAGES. >>> I'm using ALLOW_MAKE_JOBS=3Dyes already so it won't help = unfortunately, I think. >>=20 >> Those are not the same things. Check the docs. >>=20 >> Briefly, ALLOW_MAKE_JOBS will allow building several packages at once=20= >> (one per core) with one core dedicated to each. >=20 > Nope: /usr/local/etc/poudriere.conf.sample reports that it enables > more than one process per cpu: >=20 > # By default MAKE_JOBS is disabled to allow only one process per cpu > # Use the following to allow it anyway =20 > # ALLOW_MAKE_JOBS=3Dyes My wording was poor by being incomplete: ALLOW_MAKE_JOBS allows each builder to have as many processes as there are cores/processors. So if multiple builders are allowed (and happen to be in use), then the total number of processes ready to run for building can be bigger than the number of cores/processors, something that shows up in the load = averages. How many builders are allowed can be controlled with: # parallel build support. # =20 # By default poudriere uses hw.ncpu to determine the number of builders. # You can override this default by changing PARALLEL_JOBS here, or # by specifying the -J flag to bulk/testport. #=20 # Example to define PARALLEL_JOBS to one single job # PARALLEL_JOBS=3D1 Looking in /usr/ports/Mk/bsd.port.mk there is also the following, that is part of where some of the "MAKE_JOBS" terminology comes from: # MAKE_JOBS_UNSAFE # - Disallow multiple jobs even when user = set a global override. # To be used with known bad ports. # DISABLE_MAKE_JOBS # - Set to disable the multiple jobs = feature. User settable. # MAKE_JOBS_NUMBER # - Override the number of make jobs to be = used. User settable. # MAKE_JOBS_NUMBER_LIMIT # - Set a limit for maximum number of make = jobs allowed to be # used. >> ALLOW_MAKE_JOBS_PACKAGES will let a single package build using more = cores. >=20 >=20 > Even when MAKE_JOBS is otherwise disabled: >=20 > # List of packages that will always be allowed to use MAKE_JOBS =20= > # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports=20 > # which holdup the rest of the queue to build more quickly. > #ALLOW_MAKE_JOBS_PACKAGES=3D"pkg ccache py*" =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)