From owner-freebsd-fs@FreeBSD.ORG Fri Jan 1 23:17:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C5A41065692 for ; Fri, 1 Jan 2010 23:17:38 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-yw0-f197.google.com (mail-yw0-f197.google.com [209.85.211.197]) by mx1.freebsd.org (Postfix) with ESMTP id F015C8FC15 for ; Fri, 1 Jan 2010 23:17:37 +0000 (UTC) Received: by ywh35 with SMTP id 35so5082735ywh.7 for ; Fri, 01 Jan 2010 15:17:29 -0800 (PST) Received: by 10.101.189.20 with SMTP id r20mr27615904anp.191.1262387849010; Fri, 01 Jan 2010 15:17:29 -0800 (PST) Received: from ?10.0.1.198? (udp022762uds.hawaiiantel.net [72.234.79.107]) by mx.google.com with ESMTPS id 8sm5427448ywg.19.2010.01.01.15.17.26 (version=SSLv3 cipher=RC4-MD5); Fri, 01 Jan 2010 15:17:27 -0800 (PST) Date: Fri, 1 Jan 2010 13:19:33 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Ben Schumacher In-Reply-To: <9859143f0912292118h44a33961mc8207d9b943a5f1f@mail.gmail.com> Message-ID: References: <32CA2B73-3412-49DD-9401-4773CC73BED0@patpro.net> <4B3283F2.7060804@barryp.org> <3ea87f5f62bb8ba30d798d4605a64c83@localhost> <9859143f0912292118h44a33961mc8207d9b943a5f1f@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2547152148-1490549838-1262387976=:1027" Cc: freebsd-fs@freebsd.org Subject: Re: snapshot implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jan 2010 23:17:38 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2547152148-1490549838-1262387976=:1027 Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT On Tue, 29 Dec 2009, Ben Schumacher wrote: > On Sun, Dec 27, 2009 at 6:25 PM, Jeff Roberson wrote: >> It can take some time depending on fs activity on the machine.  There are >> ways to continue to optimize it within the existing infrastructure.  It only >> requires someone willing to expend the time. > > Any idea how complex of a task this is (and how much fruit it would > bear)? I've been interested in dipping my toes into some FreeBSD > kernel work, but I'm not exactly sure where to start. I honestly don't > have tons of free time to work on it (job commitments and all that), > but am curious if this is something that an experienced C programmer > would have a shot at doing having very little experience with > low-level kernel internals. (I'm used to dealing with POSIX interfaces > and not the code that implements them...) > > I've recently picked up a copy of "The Design and Implementation of > the FreeBSD OS", so I'm starting there, but I would love it if anybody > could toss me a hint or two on what some of the low-hanging fruit in > the arena might be. I've been playing with ZFS on a few boxes now, but > I've had (even with FreeBSD 8) enough unusual crashes that I'm > personally not ready to commit to using it on at least one "mission > critical" project I'm working on. That being said I'd love to be able > to do snapshots on the box without it hanging for over an hour due to > the fact that the data drive is >400GB (frankly on the small side for > some of the storage applications I've read about on this mailing > list). > > Any hints, tips, pointers would be appreciated. The daemon book is a good start. I'd say the snapshot problem might be a bit tough right out of the gate but it could be possible if you have strong fundamentals and someone experienced mentors you. Why don't you read a bit of the daemon book and see if you can follow the existing snapshot code to understand how it works. Once you feel like you have a good graps of that email me directly and we'll talk. Kirk and I discussed ways that we could speed it up dramatically by doing copy-on-write of the cgs in the allocation functions. This would take care of the considerable delay that is incurred when making a snapshot. One good first project that would introduce you somewhat to the process of kernel programming would be to add timing instrumentation to the various stages of building a snapshot. This way you can prove where the delay happens. Then you would be familiar with using timers, building custom kernels, and a little bit of the snapshot code. Cheers, Jeff > > Cheers, > Ben > --2547152148-1490549838-1262387976=:1027--