Date: Thu, 21 Mar 2019 11:53:41 +0100 From: Marcin Wojtas <mw@semihalf.com> To: Colin Percival <cperciva@tarsnap.com> Cc: =?UTF-8?Q?Micha=C5=82_Krawczyk?= <mk@semihalf.com>, "Matushevsky, Alexander" <matua@amazon.com>, freebsd-arm <freebsd-arm@freebsd.org>, "Tzalik, Guy" <gtzalik@amazon.com> Subject: Re: ARM Graviton AWS Processor (AMI Image) Message-ID: <CAPv3WKcy3udYxr3CY5DahR26_ef9OK5mAM%2BBr%2B3QHO5HtCN1Ow@mail.gmail.com> In-Reply-To: <010001699e6c9812-a0fc8d06-51d3-4252-ac06-2c1087d2c46f-000000@email.amazonses.com> References: <79CC79B9-81AF-4563-BABE-429E6A57F476@bmalum.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazonses.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazo> <1548182399.2864.0@smtp.migadu.com> <1552255580.21373.0@unrelenting.technology> <CAPv3WKdV4BcpCTj%2BC=raa_Jh2Rce5wtSSu2XaEkP=qxa4zZi7Q@mail.gmail.com> <01000169992ab14c-aa38b2f1-3f57-4805-9d8a-fa92207e4c2d-000000@email.amazonses.com> <CAJMMOfOxOQeVwYcqQfCJ0RvfmNBCkBnd=sYN3dpkMhdGUKv%2BXA@mail.gmail.com> <010001699e6c9812-a0fc8d06-51d3-4252-ac06-2c1087d2c46f-000000@email.amazonses.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Colin, Fixes pushed: https://svnweb.freebsd.org/base?view=3Drevision&revision=3D345371 https://svnweb.freebsd.org/base?view=3Drevision&revision=3D345373 Best regards, Marcin czw., 21 mar 2019 o 06:05 Colin Percival <cperciva@tarsnap.com> napisa=C5= =82(a): > > Hi Micha=C5=82, > > Thanks, those two patches fix both problems. Can you get them into > the FreeBSD tree or should I commit them? > > Colin Percival > > On 3/20/19 3:24 AM, Micha=C5=82 Krawczyk wrote: > > Hi, > > > > we have solution for two of the issues with ENA, > > > > The double resource activation can be fixed by this patch: > > https://reviews.freebsd.org/D19655 > > > > And the issue with reloading module/reconfiguring it by using ifconfig > > can be fixed by using this patch: > > https://reviews.freebsd.org/D19641 > > > > Please check if that fixes it for you. > > > > Thanks, > > Micha=C5=82 > > > > > > =C5=9Br., 20 mar 2019 o 04:35 Colin Percival <cperciva@tarsnap.com> nap= isa=C5=82(a): > >> > >> Hi ENA people, > >> > >> Any luck figuring out what was going on with the ENA driver here > >> with interrupt activation? > >> > >> Colin Percival > >> > >> On 3/11/19 1:35 AM, Marcin Wojtas wrote: > >>> +FreeBSD ENA maintainers > >>> > >>> W dniu pon., 11.03.2019 o 00:40 Greg V <greg@unrelenting.technology> > >>> napisa=C5=82(a): > >>> > >>>> > >>>> On Tue, Jan 22, 2019 at 9:39 PM, Greg V <greg@unrelenting.technology= > > >>>> wrote: > >>>>> On Mon, Jan 21, 2019 at 1:11 PM, Martin Karrer <martin@bmalum.com> > >>>>> wrote: > >>>>>> My question is if there are any plans yet to support the Graviton > >>>>>> ARM instances of AWS? > >>>>>> > >>>>>> We have a heavy load on FreeBSD and would also use the ARM > >>>>>> instances. Are there any other interested parties? > >>>>> > >>>>> I have tried this. It should work very well in theory, e.g. the > >>>>> network card driver (if_ena) compiles with no changes for aarch64, > >>>>> and in fact NetBSD has ported this driver and is up and running on > >>>>> these instances: https://dmesgd.nycbug.org/index.cgi?do=3Dview&id= =3D4623 > >>>>> > >>>>> But my result with FreeBSD was: nothing on the console after > >>>>> loader.efi hands control to the kernel. > >>>>> [=E2=80=A6] > >>>> > >>>> Hello everyone, big update: > >>>> > >>>> FreeBSD/aarch64 on Amazon EC2 a1 (AWS Graviton) instances WORKS! > >>>> > >>>> https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D4813 > >>>> > >>>> And you can try it (well, my -CURRENT build, NO WARRANTY etc) right = now: > >>>> > >>>> ami-0c2829a0b82a62ca6 in eu-west-1 (Ireland) > >>>> > >>>> ----- > >>>> > >>>> So, what I had to do / what should be done / how others can help get > >>>> this into a finished state: > >>>> > >>>> 1. Serial console: > >>>> - I fixed it: https://reviews.freebsd.org/D19507 > >>>> - (I learned some things about UARTs and their support in FreeBSD= , > >>>> should write a blog post about that) > >>>> > >>>> 2. aarch64 build configuration: > >>>> - if_ena network driver module should be enabled: > >>>> https://reviews.freebsd.org/D18372 > >>>> - NVMe driver should be enabled in the GENERIC kernel config (dev= ice > >>>> nvme, device nvd) > >>>> - BTW, why not also go with hw.nvme.use_nvd=3D"0" by default on > >>>> aarch64, IIRC that was done on powerpc64 > >>>> > >>>> 3. VM image build system: > >>>> - GPT+EFI should be used (amd64 was GPT with no EFI, and aarch64 = was > >>>> MBR with EFI (???)): https://reviews.freebsd.org/D18371 > >>>> - bsdec2-image-upload --arm64 flag should be supported: included > >>>> above ^^ > >>>> - ec2.conf: amazon-ssm-agent shouldn't be installed when building > >>>> for aarch64 TARGET, since that's written in Go, and Go isn't ported = to > >>>> FreeBSD/aarch64 yet: > >>>> > >>>> https://github.com/myfreeweb/freebsd/commit/5b530ebf7385d8320b9076cf= 84f50aad01689bc > >>>> (untested patch, I actually used an interactive shell in between the > >>>> image build commands) > >>>> - qemu-aarch64-static should be used for preinstalling pkgs when > >>>> chrooting into the image: rough version included above ^^ > >>>> > >>>> 4. ENA (Elastic Network Adapter) driver: > >>>> - it works > >>>> - except there's something funky with interrupt activation, and i= t > >>>> hits panic("Attempt to double activation of resource id: %u\n", res_= id) > >>>> (for the management IRQ) on boot, so I applied the obvious silly > >>>> workaround of "don't panic": > >>>> > >>>> https://github.com/myfreeweb/freebsd/commit/a7e7c6e48cdbdb0fdc6c4e0b= a63392262938e62c > >>>> - but still, it doesn't properly reactivate interrupts (and the b= ox > >>>> becomes unreachable over the net) after going down and up again =E2= =80=94 > >>>> guess what does that on boot? dhclient applying the big jumbo MTU = =E2=80=94 > >>>> so I set dhclient.conf to reject MTU changes: > >>>> > >>>> https://github.com/myfreeweb/freebsd/commit/03ec4d417b0b4252285baaf4= e294cc6d8c870f7f > >>>> > >>>> > >>>> Would be great if someone familiar with interrupts and stuff could h= elp > >>>> debug the ena driver and make it work without these hacks :) > >> -- > >> Colin Percival > >> Security Officer Emeritus, FreeBSD | The power to serve > >> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly para= noid > > > > > > -- > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoi= d > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPv3WKcy3udYxr3CY5DahR26_ef9OK5mAM%2BBr%2B3QHO5HtCN1Ow>