From owner-freebsd-questions@FreeBSD.ORG Thu Apr 10 14:24:18 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 140C3B8B for ; Thu, 10 Apr 2014 14:24:18 +0000 (UTC) Received: from webmail.dweimer.net (24-240-198-187.static.stls.mo.charter.com [24.240.198.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.dweimer.local", Issuer "webmail2.dweimer.local" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D5C631863 for ; Thu, 10 Apr 2014 14:24:17 +0000 (UTC) Received: from www.dweimer.net (webmail [192.168.5.2]) by webmail.dweimer.net (8.14.7/8.14.7) with ESMTP id s3AEOAEe018635 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 10 Apr 2014 09:24:10 -0500 (CDT) (envelope-from dweimer@dweimer.net) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 10 Apr 2014 09:24:10 -0500 From: dweimer To: FreeBSD Questions Subject: ZFS can't destroy snapshot, dataset busy Organization: dweimer.net Mail-Reply-To: dweimer@dweimer.net Message-ID: <5fe3d5f142b1646768b3528206e6dd36@dweimer.net> X-Sender: dweimer@dweimer.net User-Agent: Roundcube Webmail/1.0.0 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dweimer@dweimer.net List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 14:24:18 -0000 I have a server running on ZFS, with multiple jails. I use bacula to back it up, and have written a python script that finds all my ZFS datasets (except those I have listed to ignore) and creates snapshots, then mounts those snapshots to a temporary file structure, using mount -t nullfs /zfs-mountpoint/.zfs/snapshot/snapshotname /mnt/backup/... I then backup the data from the snapshots, and kick off a second script that dismounts the nullfs mounted snapshots and destroys them. I have the same script running on multiple servers without jails and everything works great. However on this server that is running 4 jails, (base OS is FreeBSD 10-RELEASE-p1, as of last night) the root file system in the jails is frequently holding the snapshots as busy so they fail to destroy, which then breaks the backups for that jail until resolved. The only resolution I have found do an unmount -f /zfs-mountpoint/.zfs/snapshot/snapshotname and then I am able to destroy the snapshot. The nullfs mounts are gone, but I haven't been able to find what process is holding onto the snapshot file system. Before I edit my script to add this step in prior to destroying the snapshots, does anybody have any ideas on how to find out whats keeping these file systems busy? I have already tried restarting the jail using jail -r and jail -c it doesn't help, its something in the base Operating System -- Thanks, Dean E. Weimer http://www.dweimer.net/