From owner-freebsd-questions@freebsd.org Fri Apr 10 19:50:44 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F008F2C7134 for ; Fri, 10 Apr 2020 19:50:44 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48zTC80vDhz4TLL for ; Fri, 10 Apr 2020 19:50:43 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 8209E5C0145; Fri, 10 Apr 2020 15:50:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 10 Apr 2020 15:50:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= subject:from:to:references:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm2; bh=w jl2na1Pyu6lBv20ACvCsu+uIbPKX4/LW8oRxUQ3200=; b=kN0vCihoO8HsUGWaj 3wp8Iu3n5S6wdEXomi/aCIAXouL6OoGq683vTZQKryzAVbI90GSDFVcsWQV2U6AP iPFDhvsCHdeaK5nRxgZP6F8xul9OwgjyWg31SUFW5enOiTcmZWsE8SXaLttZt6No WZpjkd/IWmUo2cgHpC41MB/tGqAHmy/ew00+XtdK2AIvOtGRNWS9FevlOBasaDDY smgOEWzjHRmo1vHWQd2zjIOpigRbJvg4S2SrW6IOVJ2a2/LvmV2t6kDaujA0IYc6 R+MXS2Bo8MbUC65ZPHPdMFFpZ9VqwSPxEEhPdpdyBMYRJ+nIpXap2Q+Yp6YYwYJ/ WxRkQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=wjl2na1Pyu6lBv20ACvCsu+uIbPKX4/LW8oRxUQ32 00=; b=3p2q0OOwH5E1b6TsYQJqvbO88vETZhkhp0uNWJoTwV4qumJmAP3ODz8ox PPymGp1rwBNqidW/9pMwDjwQ7q0yVIXJUpJ0p4wuUGhAs39Q6DYHhPvH/7MMRvAK 3/vpdV2oPZwzbNW+OYIOh+dC0lmthYV+/CnJy4sTwBKDZQtBdnU5xwCzZ96ym9+h zQtcf6XeYcWWuvbszaHH32kkmXY/iLgtphTxN9ExazIw3H+lNRlnufBN4gBR61Xu hwFnrk0yZ7kcnChf1FRuEdEeWIzaFo0JJGc86XC0mQJtLy9rHJWGz7fk5PDyjL1M vZv0icBT2wGLMt65HoXOdIfYehBAA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrvddvgddugedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffhvfhfkffffgggjggtgfesthekredttdefjeenucfhrhhomhepjghurhhi ucfrrghnkhhovhcuoeihphgrnhhkohhvsehfrghsthhmrghilhdrtghomheqnecukfhppe ekhedrudejgedrudelfedrudegjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpeihphgrnhhkohhvsehfrghsthhmrghilhdrtghomh X-ME-Proxy: Received: from [192.168.1.6] (unknown [85.174.193.147]) by mail.messagingengine.com (Postfix) with ESMTPA id 026F43280059; Fri, 10 Apr 2020 15:50:41 -0400 (EDT) Subject: Re: Can't remove files from a directory that was partially used by a jail From: Yuri Pankov To: Graham Perrin , freebsd-questions@freebsd.org References: <22daa8dc-cb20-6b08-b85f-161e6aca06ec@gmail.com> <425096b6-11e8-e9ab-e7c3-6dcf4a8e6b26@fastmail.com> Message-ID: <09523a27-1344-ff33-e5de-0c70745940fb@fastmail.com> Date: Fri, 10 Apr 2020 22:50:39 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <425096b6-11e8-e9ab-e7c3-6dcf4a8e6b26@fastmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 48zTC80vDhz4TLL X-Spamd-Bar: +++++++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fastmail.com header.s=fm2 header.b=kN0vCiho; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=3p2q0OOw; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (mx1.freebsd.org: domain of ypankov@fastmail.com designates 66.111.4.28 as permitted sender) smtp.mailfrom=ypankov@fastmail.com X-Spamd-Result: default: False [9.13 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[fastmail.com]; R_SPF_ALLOW(0.00)[+ip4:66.111.4.28:c]; RWL_MAILSPIKE_GOOD(0.00)[28.4.111.66.rep.mailspike.net : 127.0.0.18]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[fastmail.com:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(0.00)[fastmail.com,none]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[147.193.174.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[fastmail.com]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(5.00)[147.193.174.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.4]; R_DKIM_ALLOW(0.00)[fastmail.com:s=fm2,messagingengine.com:s=fm2]; RECEIVED_SPAMHAUS_CSS(4.00)[147.193.174.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.3]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-0.72)[-0.721,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (-9.84), ipnet: 66.111.4.0/24(-4.89), asn: 11403(-2.69), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; BAD_REP_POLICIES(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[28.4.111.66.list.dnswl.org : 127.0.5.1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.95)[0.955,0]; GREYLIST(0.00)[pass,meta] X-Spam: Yes X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2020 19:50:45 -0000 Yuri Pankov wrote: > Graham Perrin wrote: >> Below, I can't remove files to make way for (re)creation of a jail. >> >> Any suggestions? > > Tried chflags(8)? Sorry, it's chflags(1) (didn't re-check and wrote from memory assuming such utility should be in section 8). >> If it helps: at some point a few hours ago I might have accidentally >> begun (then cancelled) a bulk operation whilst using a different boot >> environment; the build began with a warning about the jail being newer >> than the host. >> >> >> root@momh167-gjp4-8570p:~ # poudriere jail -c -j head -v head -a amd64 >> -m src=/usr/src >> [00:00:00] Error: Directory /usr/local/poudriere/jails/head already >> exists >> root@momh167-gjp4-8570p:~ # poudriere jail -l >> JAILNAME VERSION ARCH METHOD TIMESTAMP PATH >> root@momh167-gjp4-8570p:~ # rm -rf /usr/local/poudriere/jails/head >> rm: /usr/local/poudriere/jails/head/libexec/ld-elf32.so.1: Operation >> not permitted >> rm: /usr/local/poudriere/jails/head/libexec/ld-elf.so.1: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/libexec: Directory not empty >> rm: /usr/local/poudriere/jails/head/sbin/init: Operation not permitted >> rm: /usr/local/poudriere/jails/head/sbin: Directory not empty >> rm: /usr/local/poudriere/jails/head/var/empty: Operation not permitted >> rm: /usr/local/poudriere/jails/head/var: Directory not empty >> rm: /usr/local/poudriere/jails/head/usr/lib32/libthr.so.3: Operation >> not permitted >> rm: /usr/local/poudriere/jails/head/usr/lib32/librt.so.1: Operation >> not permitted >> rm: /usr/local/poudriere/jails/head/usr/lib32/libcrypt.so.5: Operation >> not permitted >> rm: /usr/local/poudriere/jails/head/usr/lib32/libc.so.7: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/lib32: Directory not empty >> rm: /usr/local/poudriere/jails/head/usr/bin/login: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/crontab: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/opiepasswd: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/chpass: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/opieinfo: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/passwd: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/bin/su: Operation not permitted >> rm: /usr/local/poudriere/jails/head/usr/bin: Directory not empty >> rm: /usr/local/poudriere/jails/head/usr/lib/librt.so.1: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/usr/lib: Directory not empty >> rm: /usr/local/poudriere/jails/head/usr: Directory not empty >> rm: /usr/local/poudriere/jails/head/lib/libcrypt.so.5: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/lib/libc.so.7: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/lib/libthr.so.3: Operation not >> permitted >> rm: /usr/local/poudriere/jails/head/lib: Directory not empty >> rm: /usr/local/poudriere/jails/head: Directory not empty >> root@momh167-gjp4-8570p:~ # rm >> /usr/local/poudriere/jails/head/libexec/ld-elf32.so.1 >> override r-xr-xr-x root/wheel schg for >> /usr/local/poudriere/jails/head/libexec/ld-elf32.so.1? y >> rm: /usr/local/poudriere/jails/head/libexec/ld-elf32.so.1: Operation >> not permitted >> root@momh167-gjp4-8570p:~ # zfs list >> NAME                                           USED  AVAIL  REFER >> MOUNTPOINT >> copperbowl                                     312G   122G    88K >> /copperbowl >> copperbowl/ROOT                               94.8G   122G    88K none >> copperbowl/ROOT/Waterfox                       314M   122G  59.3G  / >> copperbowl/ROOT/r357746g                      1.22M   122G  59.8G  / >> copperbowl/ROOT/r357746h                      1.37M   122G  60.0G  / >> copperbowl/ROOT/r359628c                      19.1G   122G  60.9G  / >> copperbowl/ROOT/r359750a                      1.37M   122G  61.4G  / >> copperbowl/ROOT/r359750b                      75.0G   122G  60.7G  / >> copperbowl/ROOT/r359750c                       392M   122G  62.7G  / >> copperbowl/iocage                             4.35G   122G  3.68M >> /copperbowl/iocage >> copperbowl/iocage/download                     359M   122G    88K >> /copperbowl/iocage/download >> copperbowl/iocage/download/12.0-RELEASE        359M   122G   359M >> /copperbowl/iocage/download/12.0-RELEASE >> copperbowl/iocage/images                        88K   122G    88K >> /copperbowl/iocage/images >> copperbowl/iocage/jails                       2.75G   122G    88K >> /copperbowl/iocage/jails >> copperbowl/iocage/jails/jbrowsers             2.75G   122G    92K >> /copperbowl/iocage/jails/jbrowsers >> copperbowl/iocage/jails/jbrowsers/root        2.75G   122G  3.99G >> /copperbowl/iocage/jails/jbrowsers/root >> copperbowl/iocage/log                          100K   122G   100K >> /copperbowl/iocage/log >> copperbowl/iocage/releases                    1.24G   122G    88K >> /copperbowl/iocage/releases >> copperbowl/iocage/releases/12.0-RELEASE       1.24G   122G    88K >> /copperbowl/iocage/releases/12.0-RELEASE >> copperbowl/iocage/releases/12.0-RELEASE/root  1.24G   122G  1.24G >> /copperbowl/iocage/releases/12.0-RELEASE/root >> copperbowl/iocage/templates                     88K   122G    88K >> /copperbowl/iocage/templates >> copperbowl/poudriere                          29.6G   122G    88K >> /copperbowl/poudriere >> copperbowl/poudriere/data                     1.92G   122G    96K >> /usr/local/poudriere/data >> copperbowl/poudriere/data/.m                    88K   122G    88K >> /usr/local/poudriere/data/.m >> copperbowl/poudriere/data/cache                 88K   122G    88K >> /usr/local/poudriere/data/cache >> copperbowl/poudriere/data/logs                1.48G   122G  1.48G >> /usr/local/poudriere/data/logs >> copperbowl/poudriere/data/packages             446M   122G   446M >> /usr/local/poudriere/data/packages >> copperbowl/poudriere/data/wrkdirs               88K   122G    88K >> /usr/local/poudriere/data/wrkdirs >> copperbowl/poudriere/jails                      88K   122G    88K >> /copperbowl/poudriere/jails >> copperbowl/poudriere/ports                    27.7G   122G    88K >> /copperbowl/poudriere/ports >> copperbowl/poudriere/ports/default            27.7G   122G  27.7G >> /usr/local/poudriere/ports/default >> copperbowl/tmp                                 456K   122G   456K /tmp >> copperbowl/usr                                 179G   122G    88K /usr >> copperbowl/usr/home                            175G   122G   159G >> /usr/home >> copperbowl/usr/ports                          1.90G   122G  1.90G >> /usr/ports >> copperbowl/usr/src                            2.09G   122G  2.09G >> /usr/src >> copperbowl/var                                4.65G   122G    88K /var >> copperbowl/var/audit                            88K   122G    88K >> /var/audit >> copperbowl/var/crash                          1.27M   122G  1.27M >> /var/crash >> copperbowl/var/log                            14.5M   122G  14.5M >> /var/log >> copperbowl/var/mail                            128K   122G   128K >> /var/mail >> copperbowl/var/tmp                            4.64G   122G  4.64G >> /var/tmp >> root@momh167-gjp4-8570p:~ #