Date: Sun, 10 Apr 2016 10:39:54 -0400 From: PK1048 <paul@pk1048.com> To: Wim Lewis <wiml@omnigroup.com> Cc: "freebsd-fs@FreeBSD.org" <freebsd-fs@freebsd.org> Subject: Re: ZFS pool with a large number of filesystems Message-ID: <A677620F-7787-4EED-AA94-48887965F58F@pk1048.com> In-Reply-To: <34DB45E8-7E1F-4D7C-96FF-E0A403EE8000@omnigroup.com> References: <34DB45E8-7E1F-4D7C-96FF-E0A403EE8000@omnigroup.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Apr 4, 2016, at 20:38, Wim Lewis <wiml@omnigroup.com> wrote: >=20 > I'm curious how many ZFS filesystems are reasonable to have on a = single machine (in a single zpool). We're contemplating a design in = which we'd have tens of thousands, perhaps a couple hundred thousand, = filesystems mounted out of the same pool. Before we go too far into = investigating this idea: Does anyone have real-world experience doing = something like that? Is it a situation that ZFS-on-FreeBSD is engineered = to handle with good performance? Is there a rough estimate of the = resources consumed per additional filesystem (in terms of kernel VM and = disk space)? >=20 > Thanks for any insight or advice (even, or especially, if the answer = is "that's crazy, don't do that" :) ) In a previous life I ran ZFS on Solaris (10) with hundreds of datasets, = but all off of the top level zpool. We also had hundreds of thousands of = snapshots. There was no noticeable hit to user performance, but as = others have pointed out, getting a listing of all the datasets and = snapshots was not a fast operation. zfs list -t filesystem on the other = hand was fast. Fast forwarding to today=E2=80=A6 I have a handful of FreeBSD boxes = running ZFS with less than 100 datasets but with _hourly_ snapshots kept = for 30 days. That is 720 snapshots per dataset. Our largest server has = about 50 datasets, so about 36,000 snapshots. zfs list -t filesystem is = fast while zfs list -t snapshot is slow, but I don=E2=80=99t do that = very often. No performance hit to normal I/O. The FreeBSD hardware is Xeon X55xx or E55xx, usually 2 quad core CPUs, = so 8 or 16 vCPU. RAM varies from low of 12GB to a high of 80GB. Systems = are running a mix of vBox VMs, file service, backup service (including = Mac OS Timemachine backups via NetATalk).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A677620F-7787-4EED-AA94-48887965F58F>