Date: Mon, 11 Mar 2019 09:35:11 +0100 From: Marcin Wojtas <mw@semihalf.com> To: Greg V <greg@unrelenting.technology> Cc: Martin Karrer <martin@bmalum.com>, "Matushevsky, Alexander" <matua@amazon.com>, =?UTF-8?Q?Micha=C5=82_Krawczyk?= <mk@semihalf.com>, =?UTF-8?B?UmFmYcWCIEtvemlr?= <rk@semihalf.com>, freebsd-arm@freebsd.org, freebsd-cloud@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: ARM Graviton AWS Processor (AMI Image) Message-ID: <CAPv3WKdV4BcpCTj%2BC=raa_Jh2Rce5wtSSu2XaEkP=qxa4zZi7Q@mail.gmail.com> In-Reply-To: <1552255580.21373.0@unrelenting.technology> References: <79CC79B9-81AF-4563-BABE-429E6A57F476@bmalum.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazonses.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazo> <1548182399.2864.0@smtp.migadu.com> <1552255580.21373.0@unrelenting.technology>
next in thread | previous in thread | raw e-mail | index | archive | help
+FreeBSD ENA maintainers W dniu pon., 11.03.2019 o 00:40 Greg V <greg@unrelenting.technology> napisa=C5=82(a): > > On Tue, Jan 22, 2019 at 9:39 PM, Greg V <greg@unrelenting.technology> > wrote: > > On Mon, Jan 21, 2019 at 1:11 PM, Martin Karrer <martin@bmalum.com> > > wrote: > >> My question is if there are any plans yet to support the Graviton > >> ARM instances of AWS? > >> > >> We have a heavy load on FreeBSD and would also use the ARM > >> instances. Are there any other interested parties? > > > > I have tried this. It should work very well in theory, e.g. the > > network card driver (if_ena) compiles with no changes for aarch64, > > and in fact NetBSD has ported this driver and is up and running on > > these instances: https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D462= 3 > > > > But my result with FreeBSD was: nothing on the console after > > loader.efi hands control to the kernel. > > [=E2=80=A6] > > Hello everyone, big update: > > FreeBSD/aarch64 on Amazon EC2 a1 (AWS Graviton) instances WORKS! > > https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D4813 > > And you can try it (well, my -CURRENT build, NO WARRANTY etc) right now: > > ami-0c2829a0b82a62ca6 in eu-west-1 (Ireland) > > ----- > > So, what I had to do / what should be done / how others can help get > this into a finished state: > > 1. Serial console: > - I fixed it: https://reviews.freebsd.org/D19507 > - (I learned some things about UARTs and their support in FreeBSD, > should write a blog post about that) > > 2. aarch64 build configuration: > - if_ena network driver module should be enabled: > https://reviews.freebsd.org/D18372 > - NVMe driver should be enabled in the GENERIC kernel config (device > nvme, device nvd) > - BTW, why not also go with hw.nvme.use_nvd=3D"0" by default on > aarch64, IIRC that was done on powerpc64 > > 3. VM image build system: > - GPT+EFI should be used (amd64 was GPT with no EFI, and aarch64 was > MBR with EFI (???)): https://reviews.freebsd.org/D18371 > - bsdec2-image-upload --arm64 flag should be supported: included > above ^^ > - ec2.conf: amazon-ssm-agent shouldn't be installed when building > for aarch64 TARGET, since that's written in Go, and Go isn't ported to > FreeBSD/aarch64 yet: > > https://github.com/myfreeweb/freebsd/commit/5b530ebf7385d8320b9076cf84f50= aad01689bc > (untested patch, I actually used an interactive shell in between the > image build commands) > - qemu-aarch64-static should be used for preinstalling pkgs when > chrooting into the image: rough version included above ^^ > > 4. ENA (Elastic Network Adapter) driver: > - it works > - except there's something funky with interrupt activation, and it > hits panic("Attempt to double activation of resource id: %u\n", res_id) > (for the management IRQ) on boot, so I applied the obvious silly > workaround of "don't panic": > > https://github.com/myfreeweb/freebsd/commit/a7e7c6e48cdbdb0fdc6c4e0ba6339= 2262938e62c > - but still, it doesn't properly reactivate interrupts (and the box > becomes unreachable over the net) after going down and up again =E2=80=94 > guess what does that on boot? dhclient applying the big jumbo MTU =E2=80= =94 > so I set dhclient.conf to reject MTU changes: > > https://github.com/myfreeweb/freebsd/commit/03ec4d417b0b4252285baaf4e294c= c6d8c870f7f > > > Would be great if someone familiar with interrupts and stuff could help > debug the ena driver and make it work without these hacks :) > > > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPv3WKdV4BcpCTj%2BC=raa_Jh2Rce5wtSSu2XaEkP=qxa4zZi7Q>