n, 20 May 2024 05:20:10 +0000 (UTC) (envelope-from tatsuki_makino@hotmail.com) Received: from SINPR02CU002.outbound.protection.outlook.com (mail-southeastasiaazolkn19011003.outbound.protection.outlook.com [52.103.65.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VjQqs072nz4QKj for ; Mon, 20 May 2024 05:20:08 +0000 (UTC) (envelope-from tatsuki_makino@hotmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hotmail.com header.s=selector1 header.b=Mn5cdsYV; dmarc=pass (policy=none) header.from=hotmail.com; spf=pass (mx1.freebsd.org: domain of tatsuki_makino@hotmail.com designates 52.103.65.3 as permitted sender) smtp.mailfrom=tatsuki_makino@hotmail.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JiB8O2AdfSeCCMtgoOcIv6yF54bLrX7G6vB9hwkktjoVkX+XapX0adxbiTlebd2OL8UdyLSQJU4PDIXNWpoplfc387+g5y9CH73o9hZd8EGf7Ilsq6KsVeePcigHl71z3tQ8ccAMQhuwEINGTYAMRzgW1IS6bmZm3pEuyTExVQ+cCjbH1yxwZ5G6SHorox6GR1WS45pJ6WnbWcrCxxGoqgKd3UbRNYrnyHTuiMYq5fA/u2J3KbncOBqCzRxdbo4+2MV9uRavGOkopPjqkhkIpkfmg/8jdCauHsOcQGA5c1eUHakrlaGi0gr5EfBeKd78c+lWMJ36VNjWSWV2bDq2AA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xTB5eMykxTtzpr9V3soo2q1nZnkr2Ppj9FW1UYDco6g=; b=OYtcHqJhHUUgG4NaPzWLcun6FGGXKGXjwRv3QezAUNqF77IUMXqYMkoe+nRGZEODKcszk0M6yt9J02HExWwAV0wIkuILuLv8WaRqBxSUjZHLj/ZRVq0GzuItkynJWkYdShZu0lXyKWWOyE1h8u8FSqCGmeECy+cJLOM/eakz1gmvUebo23bEk7ktEW3GlhnQFXi9+m5rnqtNEpCsJisPhYDEDdashhk8xmA4TlxGjyb6Zh0Qxj/mLkUBFTFZUMQrwDNHpF3zxbbJL4MXyMvRylZjlhinXTRar2GqtH/B6cBNMhofZnyN1SUxsn+flfGVBCM9m2Bo/qxc4U+R1cAEFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xTB5eMykxTtzpr9V3soo2q1nZnkr2Ppj9FW1UYDco6g=; b=Mn5cdsYVaO7X6mJ5dO/GD4sk8mMY+lhVAiYkcV3wIiDWWVBVhIoQ1j0lQatd8HTsoeLShIEGVD8bUZzpFi5M/8lj7LCJRSXhhx4XTEFRktnIY2WnfYmU372u1daShnAXuTWchztf5F5dN7/YmmRqeCzEwlEdCSjgU6hBMZwHUrFOf8jpc0z1iA09Tj0e4G087NBP5HWiyrCOFm6jALX6e9ORSXltt7A9Dllb+c5JAKWYZkJk9bFqAw67hMmoeCpYBWFBCeyQFQMLX+ndCZ6R+ZB3J8uc41mSyQOrI4XlP8QFHJR3ue4x7dl0RmW3OmvLlQVOArPonRWKK4qFymQaFA== Received: from SI2PR01MB5036.apcprd01.prod.exchangelabs.com (2603:1096:4:1f8::9) by SEZPR01MB5222.apcprd01.prod.exchangelabs.com (2603:1096:101:eb::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May 2024 05:20:03 +0000 Received: from SI2PR01MB5036.apcprd01.prod.exchangelabs.com ([fe80::546c:7ecf:524e:4c34]) by SI2PR01MB5036.apcprd01.prod.exchangelabs.com ([fe80::546c:7ecf:524e:4c34%4]) with mapi id 15.20.7587.030; Mon, 20 May 2024 05:20:02 +0000 Subject: Re: I've started collecting tmpfs usage figures from a poudriere-devel bulk -a for later publishing some of the top ones (handy for TMPFS_BLACKLIST judgments) From: Tatsuki Makino To: freebsd-ports@FreeBSD.org References: <97E1D1BA-CBA1-42C6-8202-B6D654941C28.ref@yahoo.com> <97E1D1BA-CBA1-42C6-8202-B6D654941C28@yahoo.com> Message-ID: Date: Mon, 20 May 2024 14:19:44 +0900 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.4 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-TMN: [bPNvSvyExDXlUdMjlFzjRi5miRWiZUoO] X-ClientProxiedBy: TYAPR01CA0198.jpnprd01.prod.outlook.com (2603:1096:404:29::18) To SI2PR01MB5036.apcprd01.prod.exchangelabs.com (2603:1096:4:1f8::9) X-Microsoft-Original-Message-ID: <310a1eb4-a394-5f40-7588-2eece1d35c8d@hotmail.com> List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-ports@freebsd.org Sender: owner-freebsd-ports@FreeBSD.org MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR01MB5036:EE_|SEZPR01MB5222:EE_ X-MS-Office365-Filtering-Correlation-Id: 5c298cb8-a023-42bc-2dc0-08dc788c803a X-Microsoft-Antispam: BCL:0;ARA:14566002|461199019|3412199016|440099019; X-Microsoft-Antispam-Message-Info: Y6l+uXxNqdWaxW7DtBJXcvKNYB8Bb/+NFZKIrT2ipF8c80G/r4kKQ+Ii9sYLXHkiDA8jMcvpQ9qonAXKbny4Z6CzOAqeWqGQbcnw0iAqJJvY1N10rdAiCUVkj9LnSbTnU0WHwlz29aD+MvTjz0vaJGWjsFdKEYoc6idKCwYXMngBMixYHXo5ONVdQ+C0BrPJgDmuRUuNDmgNaaDzBdFIHuHfkaKEqB/qoJm2xLwlM2iXmBWzejRiG3gexHv7e4DwZ3shGjI+4K0dQBtDNrqmlwTSHRwH8bcqHgZr31av1ZwCt2EvaYTs/f+iqPe9cTNhvQX7lnXPl1V7C6Jq9fOn7+GKB+JP3Rmn1nPqL83pXlg4GQJDytpYsuDqWRDJChl+/26hDLNuog9Jq7lf57RQjpuBH+SELIqWKTy5C7SqiYs2pxbaZuvA0jvG1JoOVeTZOL0oODNtj4bTvj1wChjcfhA0bGLRE+RmaKvYEnR8oVlC2qR4i++4Pkz8GMUoGoVI+Y85CUwJLuKPaDQrnlu+cyFq8g99ywGcQd5lkqYsHmiKVk1GNX7G1fXsbqRjLiZJ X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a0NDQ0FjME5XUmVqVkpONkRyeWZ5SG5OeUtVbThwZW56bkpwd3dDWU5aaHN3?= =?utf-8?B?WW1hZzIwUjNRc09jWDlnY1gwZEJWOU1jOXB4U1JyNktIWVppNkZsZGx6MXcz?= =?utf-8?B?T041Vks2TkI1a21TSmE0ZFRrNm1UUkpBclg0K0VNb040TFdFK2VzS1ZNajdS?= =?utf-8?B?Q3A5cURURnVSL0luVEFTRERveUdTUFE2eFpxV0taaDhzd1NOWjYxR0I5WjdW?= =?utf-8?B?Z3lBbDFNbVlDcE14VTR0ZzNydVcxVUdTSk5QYTJkS1paUmJGTTkrWExOZzdU?= =?utf-8?B?OUpacmpkK1ozeEpsbWcrL3k2anJ6MmxmTHZRWDlzSkU4RjBsVXRLK2R2SUdQ?= =?utf-8?B?c1dTS2JwQVNVME5ZeGx3U0NreTVRczlLS29QSXp2SEVHdFRBTDQxaHhUOHFs?= =?utf-8?B?ZzRZb21oMnlSTHZzem9ROGQ2NVlLVWlGTU56dmhMcDdQZTNZTmNQaVVHcWVG?= =?utf-8?B?NTlvMGRqK1IrdENKYmxhVUcwdEN5NXNBQXlhNWgvQ0V6Q3ZNeEs1Y2Q2OTdH?= =?utf-8?B?NUQvMkE5TG9GUGF3Z1F3UWRNQjRFd3IySzlTTTgzZ3BFTGVkWnFOKzNzWjRJ?= =?utf-8?B?QnkxVEdHcENWVGs2aVpSdzdkc1pvK1NpSnN2dWR5eGdyblppcGVkR2lXTXZs?= =?utf-8?B?NFFDQXVUVDZLS1NpeEc3SVBUTUYyTmdodGhQckpwVnlvLzR4Y1FVNWZnaEVw?= =?utf-8?B?eEpCeDJmT3VPQkxCNERXbDB0TzJHZXBiVTRXc3VNUHdxRjVvL29aOUdpVUw3?= =?utf-8?B?amFydDBjNzFES052dEtoUzE5djAySE5TTGRIMGplcitCQW9MLzNBcjlrTlli?= =?utf-8?B?T2tSRWxqVUFmcldaZ3liZlpYQU1UMW50MVBBNys2ZTIxSnZDQ1d5Qy9HV3RK?= =?utf-8?B?SVd6S1J5bWlXdjV0M20ycW9JTi9KWU9wNzBYK2NqYXVMOWxtWWpDeGRudElP?= =?utf-8?B?d1lVQm4xOEgvdjVGSG52Mnp3TzV3ZVlZazNYMEZ1UEpKUXdQV1BYQkpDSTYr?= =?utf-8?B?NE1XWHloWTcweVkvNllyK2JuOXhERjlPYmlHYy9ONEh1YVY5K2VxcHlML1ZU?= =?utf-8?B?aHd6dElNKzlVNzBtdEQvUVYxV1hTYXpPR0pFa2hnamNxY2VhZ0hHcUEvTW83?= =?utf-8?B?NndyUVR4MGpzVFJGaHp1L2h1a3F4Y0JEb2VYL3ZEeDRnUFdGY1FSQmhZVjV6?= =?utf-8?B?cURTMVhBMkFxQ3I4RlVhVytjcU8raGZwTFZwdXRoQTczWXFYTm5TQUxtbHlv?= =?utf-8?B?TXRRV2F6WDhLTU5UWWdnNklvdkswcUVVUmhPbEdSQnJxTENoSlRTeURqT3lk?= =?utf-8?B?Q1Y5c2F6b3piaGdhcjFWVEJnUHhxSjFvTTFLZjlWQnk0UkVLdm5qRVV5RWNU?= =?utf-8?B?RlVjYWRrZGVIbVZiU0hoYk1PcEI1RVpxUXBQd0JheVY2K2FFRHNSSlZBSzM0?= =?utf-8?B?RGNEcWFQK0g2U0U1QmtVckkvdzRvUHhoS3BHTWJXbjVSc3dUb0EwTlZkTytW?= =?utf-8?B?cGlLYzc4amVENzFQcFA2NjViOXFLQmxRQU5kQyt6Q2o5M2RWeWltY2N4Wldm?= =?utf-8?B?cEYwd0phdUVTK2xrZklLMVNtNHErTkxhUlRhSEl0V0psWHRUbkxCaU5kRHcy?= =?utf-8?B?MXlIN2U5d3dudDk3OHpxZVB4OXpOeVExeEsvUmthSkVrWG5wcXY4cTFoWFI5?= =?utf-8?Q?7iJBHneS6b4+NPjkwJRE?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-d8e84.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 5c298cb8-a023-42bc-2dc0-08dc788c803a X-MS-Exchange-CrossTenant-AuthSource: SI2PR01MB5036.apcprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 05:20:02.5121 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR01MB5222 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FORGED_MUA_SEAMONKEY_MSGID_UNKNOWN(2.50)[]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[hotmail.com,none]; R_SPF_ALLOW(-0.20)[+ip4:52.103.0.0/17]; R_DKIM_ALLOW(-0.20)[hotmail.com:s=selector1]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_FROM(0.00)[hotmail.com]; ASN(0.00)[asn:8075, ipnet:52.96.0.0/12, country:US]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[hotmail.com]; RCVD_COUNT_TWO(0.00)[2]; RWL_MAILSPIKE_POSSIBLE(0.00)[52.103.65.3:from]; RCVD_IN_DNSWL_NONE(0.00)[52.103.65.3:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@FreeBSD.org]; DWL_DNSWL_NONE(0.00)[hotmail.com:dkim]; DKIM_TRACE(0.00)[hotmail.com:+] X-Rspamd-Queue-Id: 4VjQqs072nz4QKj Hello, especially those who add some kind of tweaks to poudriere :) In my case, I try to make it a hook script first, so it is not affected by errexit of shell. However, when modifying poudriere to incorporate the codes, some tmpfs are not mounted depending on USE_TMPFS value, so poudriere exits with an error when trying to handle them. So I will post a revised version of the hook script that includes ideas to counteract that, including this one. The file should be placed in a location that will be erased by poudriere-logclean. That would require another step to collect the results... I don't know if multiple concurrent executions of it will occur, but even if it does, it has been changed to work exclusively. And I am trying to use jq to get and add numbers. If you see a point that you think is an interesting idea, cut it out and use it :) Regards. Below is the script. #!/bin/sh case "${0##*/}" in pkgbuild.sh) case "${1}" in # ( success|failed) if ! [ ${LOCKF_HOOK_TMPFS:+1} ] ; then shopt="${-}" LOCKF_HOOK_TMPFS=1 lockf -k "${LOG}/.tmpfs.log" /bin/sh ${shopt:+"-${-}"} "${0}" ${1+"${@}"} elif [ ${MY_JOBID:+1} ] ; then # jq was installed by package textproc/jq if JQ=$(which jq 2>&${OUTPUT_REDIRECTED_STDERR:-2}) ; then realpath=$(realpath "${MASTERMNT}/../${MY_JOBID}/") pour_xo_sauce_on_df=$( mount -t tmpfs -p | cut -f 2 -w | grep -e "^${realpath}" | xargs df -b --libxo json | ${JQ} -c -r '[.["storage-system-information"].filesystem[].["used-blocks"]]|add//-0' ) tmpfs_at_end=$( echo "${pour_xo_sauce_on_df}" | awk '{ x += $0; } END { printf "%s %.2f %s", "TMPFS:", x*512/(1024**3), "GiB" }' ) elif ! tmpfs_at_end="$(env BLOCKSIZE=512 df -t tmpfs \ ${MASTERMNT}/../${MY_JOBID}/ \ ${MASTERMNT}/../${MY_JOBID}/.p/ \ ${MASTERMNT}/../${MY_JOBID}/usr/local/ \ 2>/dev/null | tail -3 \ | awk '{ tmpfs_use += $3; } END { printf "%s %.2f %s", "TMPFS:", tmpfs_use*512/(1024**3), "GiB" }')" ; then tmpfs_at_end="TMPFS: -0. GiB" fi if true ; then # output to stdout echo "${2} | ${3}: ${tmpfs_at_end}" >&${OUTPUT_REDIRECTED_STDOUT:-1} fi if false ; then # output to stderr echo "${2} | ${3}: ${tmpfs_at_end}" >&${OUTPUT_REDIRECTED_STDERR:-2} fi if true && [ ${OUTPUT_REDIRECTED:-0} -eq 1 ] ; then # output to ${LOG}/logs/${3}.log file echo "${tmpfs_at_end}" >&2 fi if true ; then # append to file echo "${2} | ${3}: ${tmpfs_at_end}" >>"${LOG}/.tmpfs.log" fi fi ;; # ( *) esac ;; # ( *) esac return 0 # end of script