From owner-freebsd-stable@FreeBSD.ORG Sun Aug 7 09:22:45 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E75CA1065679 for ; Sun, 7 Aug 2011 09:22:45 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A1DE78FC14 for ; Sun, 7 Aug 2011 09:22:45 +0000 (UTC) Received: by vws18 with SMTP id 18so1854370vws.13 for ; Sun, 07 Aug 2011 02:22:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=InoLCgKbn09IRaXCGS1mfrBUsEWJI2loGpc9Ffj8SOg=; b=Rgktw4npEMdITLBKJueTMcENNiKE6NdLpwn2Ln/6E4sqBrdTVeUyyuQ9eA3suKc6Bt u0F/duAjNOqKZL2hqZrfToeUYWGJFv8iQ4+afvnG1MQZeJAM9An+/PAwthWWkEZ5yHQ5 siGX09DGQ9P41FGFuy4ny4IUGW6+3f1Lj/Z3M= MIME-Version: 1.0 Received: by 10.52.172.228 with SMTP id bf4mr986274vdc.264.1312708964490; Sun, 07 Aug 2011 02:22:44 -0700 (PDT) Received: by 10.52.169.6 with HTTP; Sun, 7 Aug 2011 02:22:44 -0700 (PDT) In-Reply-To: <010350C0-B3B3-44FC-8D94-A111C579860C@gmail.com> References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> <4E3CBB74.9020208@FreeBSD.org> <20110806062415.GB88904@in-addr.com> <010350C0-B3B3-44FC-8D94-A111C579860C@gmail.com> Date: Sun, 7 Aug 2011 10:22:44 +0100 Message-ID: From: "seanrees@gmail.com" To: Gary Palmer Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Doug Barton , freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2011 09:22:46 -0000 On Sun, Aug 7, 2011 at 10:20 AM, Sean Rees wrote: > > On Aug 6, 2011, at 07:24, Gary Palmer wrote: > >> On Fri, Aug 05, 2011 at 08:56:36PM -0700, Doug Barton wrote: >>> On 08/05/2011 20:38, Daniel O'Connor wrote: >>> >>>> Ahh, but OP had moved these files away and performance was still poor.= . _that_ is the bug. >>> >>> I'm no file system expert, but it seems to me the key questions are; ho= w >>> long does it take the system to recover from this condition, and if it'= s >>> more than N $periods is that a problem? We can't stop users from doing >>> wacky stuff, but the system should be robust in the face of this. >> >> Its been quite a while since I worked on the filesystem stuff in any >> detail but I believe, at least for UFS, it doesn't GC the directory, >> just truncate it if enough of the entries at the end are deleted >> to free up at least one fragment or block. =A0If you create N files and >> then a directory and move the N files into the directory, the directory >> entry will still be N+1 records into the directory and the only way to >> "recover" is to recreate the directory that formerly contained the N >> files. =A0It is theoretically possible to compat the directory but since >> the code to do that wasn't written when I last worked with UFS I suspect >> its non trivial. >> >> I don't know what ZFS does in this situation > > It sounds like it does something similar. > > I re-ran the experiment to see if I could narrow down the problem. > > % mkdir foo > % cd foo && for i in {1..1000}; do touch $i; done Self-pedant mode enabled: for i in {1..1000000} :) I truncated the zeros in correcting the copy/paste from my shell :) Sean