Date: Sat, 25 Nov 2017 23:55:50 +0000 From: Colin Percival <cperciva@tarsnap.com> To: Rafal Lukawiecki <raf@rafal.net>, freebsd-cloud@freebsd.org Subject: Re: FreeBSD AWS AMI disk system Message-ID: <0100015ff59bd0cd-42713e64-05b2-42e6-a964-ccaafb1d2a28-000000@email.amazonses.com> In-Reply-To: <386518E3-1D01-4E1F-BB77-E9C530E05381@rafal.net> References: <386518E3-1D01-4E1F-BB77-E9C530E05381@rafal.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/25/17 04:24, Rafal Lukawiecki wrote: > Ideally, I would like to have the simplicity of using the official AMIs but > have the option of selecting a different initial volume size (both smaller, > 4-5GB, and larger) This is something you can already do. The "default" volume size is 10 GB, but the actual disk image is 4 GB. (I think. It was 2 GB but has increased over the years.) When FreeBSD first boots in EC2, it automatically grows its root filesystem to fill the disk it's launched with. One minor issue: At one point there was a bug in the way that Marketplace images were handled which prevented launching with a disk smaller than 10 GB; I don't know if this has been fixed yet. But you can still launch with larger disks. > and to turn on the standard EBS encryption at the > initial instance launch time. Unless things have changed in how EBS works, this isn't possible. When you launch an EC2 instance, the newly created EBS volume isn't fully initialized yet -- that would slow things down far too much. Instead, EBS keeps track of which blocks haven't been initialized and loads data from the backing snapshot as needed. But if a disk is "encrypted", EBS will expect to load encrypted blocks from the snapshot stored in S3 -- AFAIK it can't handle having some of the backing blocks being encrypted and some of them being unencrypted. So (again, unless things have changed) if you want an encrypted volume you're going to have to launch it from an encrypted snapshot, meaning that you'll have to copy the AMI. > Then there is the question of the actual file system. Have you opinions > about any performance gains, especially startup/reboot time, for OpenZFS > via EBS? The usual ZFS advantages of versioning/ZFS snapshots and the > ability to stream updates seem attractive to our way of running our > (growing) server farm. I was just reading the current issue of the FreeBSD > Magazine and I have found out that ScaleEngine use OpenZFS in their AWS > set-up. I haven't measured performance for UFS vs. ZFS; I suspect that any differences will be insignificant given how fast the disks are. So far I've stuck with UFS images mainly because I expect that people who use ZFS will probably want to create a ZFS pool out of additional volumes they attach -- not out of the single 10 GB (by default) disk which has the base OS. > As an aside, the EFS is pretty slow—I suppose I had higher expectations, > considering EBS performance. I do not think that has much to do with > FreeBSD and more likely a limitation of AWS NFS, but I wonder if there is > anything on the horizon that could improve it. Assuming you're not running into the EFS I/O throttling, I suspect you're just seeing "NFS is slower than local filesystems". > On another note, will the > >32 bit NFS log spam disappear anytime soon, ie. when is FreeBSD likely to > get 64-bit handles? That's fixed in HEAD, but won't be MFCed because 64-bit inodes break all sorts of interfaces. So, 12.0 (aimed for early 2019, I believe). -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0100015ff59bd0cd-42713e64-05b2-42e6-a964-ccaafb1d2a28-000000>