From owner-freebsd-arm@freebsd.org Fri Mar 25 18:15:03 2016 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB786ADD834 for ; Fri, 25 Mar 2016 18:15:03 +0000 (UTC) (envelope-from sylgar@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2691212F1 for ; Fri, 25 Mar 2016 18:15:03 +0000 (UTC) (envelope-from sylgar@gmail.com) Received: by mail-wm0-x234.google.com with SMTP id l68so30738543wml.1 for ; Fri, 25 Mar 2016 11:15:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=CGIzn7eboh4kaEGbWIsPKCgt2Rg1XnxgIsXOXD3NMJM=; b=t+CN2HnYwRQVBQ9hbxGCw13qYkG1jW+rAK8WrdARnV93Fq8K4NNHcKQCAOmJqJf/cl GvC0afd+AM+iPUZH/dyV/db68qyrD8y6I5Rsl+DUrA+fbfRVClhb8XKVZzXEHvyR7Key Kzihhu6Lcu0n7+MDMWYxTXphLKNhd8Ip1ZJ0/El6ndptNqkbDDS45mnN7fn5PYwZiXYb awgrVXLAaFW2PEKc1jPxVQ9bh7oSr0QIdZlWBYTmF2xk0k5IzZWroYyHaLGd+LIA4lrR lQ3NNQjB2s9h3EoPBDcs7iVJipoFKwGXDx1YoHEIuhvhTgqRTBldSatkVwg2nqJLQlKC Jlzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=CGIzn7eboh4kaEGbWIsPKCgt2Rg1XnxgIsXOXD3NMJM=; b=lj0my5eD/D+Wl3KN4H+wzz867KmFIp/y/KwzdGZGKalOwFd4djj7RmzYi4O/GDa0ae Yqn1tzAvcWFi52zYpjBIWs4TUrxMGN+8yHN/OczIKeEPIvLEeHzWOxfhV/1Qk1ihGpQL ILhwSg0tO/7mKhg+W2USuc60FsQRV/kP2EBKqZEtYu/nVh9IeMIl949LSWRnqSqTbG99 PUeWkRC8ONXWk7dfdhZDDW+KDiF6Hed+9evHqoS/2twJuSuw3U2Ayr+r9M3x6vhuj1Rt gTHV2Y6n+seTZbjwejL/jB57iyF3BqFwtNDviJ17dlVHieddQhHGJIBayvBfeOIjlbcN qJ1Q== X-Gm-Message-State: AD7BkJIu8++Sgd/f943og1qE8ntBtZW/xhpW/u8OrYoEN+2xq9PDjFw0r1PjkwF6eTp0cQ== X-Received: by 10.194.143.8 with SMTP id sa8mr18257767wjb.64.1458929700976; Fri, 25 Mar 2016 11:15:00 -0700 (PDT) Received: from [192.168.0.4] ([81.56.235.196]) by smtp.gmail.com with ESMTPSA id p189sm4138237wmb.7.2016.03.25.11.14.59 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 25 Mar 2016 11:14:59 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Booting kernel.bin directly on Raspberry Pi / external DTB support From: Sylvain Garrigues In-Reply-To: Date: Fri, 25 Mar 2016 19:14:58 +0100 Cc: ticso@cicely.de, freebsd-arm Message-Id: <2A98ACB7-2815-4169-951A-0123C71D9987@gmail.com> References: <1CCA59DC-5539-4CFB-81BA-0112E2120B3B@gmail.com> <20160325052509.GE48704@cicely7.cicely.de> <97BB142C-D335-469B-9514-D1210E869C5D@gmail.com> To: Warner Losh X-Mailer: Apple Mail (2.3124) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Mar 2016 18:15:03 -0000 u-boot> fatload mmc 0 0x100000 kernel.bin takes around the same time than ubldr (actually slightly less, let=E2=80=99= s say 4s - but that=E2=80=99s still so slow...) Happy to help or test anything to get FreeBSD=E2=80=99s kernel first = line appear on the screen near =C2=AB instantaneously =C2=BB (~ 1s) like = our NetBSD or Linux friends. Sylvain > Le 25 mars 2016 =C3=A0 19:01, Warner Losh a =C3=A9crit = : >=20 >=20 >=20 > On Fri, Mar 25, 2016 at 11:55 AM, Sylvain Garrigues > wrote: > Thank you.=20 >=20 > I wanted to boot the kernel directly so as to: > 1/ remove the dependency on a 3rd party loader > 2/ reduce boot time and make it comparable with NetBSD and Linux = (which don=E2=80=99t use u-boot) >=20 > NetBSD and Linux on the Pi boot so much faster than our FreeBSD image: = less than 1 second after power-up, the first kernel copyright line = appears on my LCD. For the FreeBSD images, the longest part is ubldr = loading the kernel. It takes around 4/5 seconds (with the same SD card). >=20 > I used recent u-boot versions (the current u-boot-rpi2 port is getting = old and is not in sync with the u-boot-rpi port recently updated), tried = with dcache enabled, but the boot process still takes several seconds, = most of the time being spent in ubldr loading the kernel. >=20 > I have no idea how we could improve the booting time? I wanted to = experiment with booting the kernel directly as the first ARM program on = my Pi, but as you explained I need some init code. I managed to compile = Andrew=E2=80=99s one = (https://github.com/freebsd/freebsd/commit/074d37d46c3f9b282cd2d849d997b1b= 39acd710c = ) but I don=E2=80=99t know how to use it. Andrew, if you read = this ;-) >=20 > I wonder if the loader could be more efficient at loading off the SD = card. The SD cards can do 10-20MB/s easily, and the ARM kernel is around = 5MB last time I checked. 5 seconds is 1MB/s, which is much slower than = we know the hardware can do. I don't know if that's because the callback = mechanism in u-boot.bin is so slow (that's what ubldr uses to load the = kernel), or there's something inherently slow about our loader. Some = analysis here might be quite useful. I'm guessing that we're not getting = all the benefits from streaming read mode because we're doing I/Os that = are tiny for some reason. But I haven't looked to make sure. >=20 > What's the speed when booting directly from u-boot.bin? >=20 > Warner >=20 >=20 >=20 > =20 >> Le 25 mars 2016 =C3=A0 06:48, Warner Losh > a =C3=A9crit : >>=20 >>=20 >>=20 >> On Thu, Mar 24, 2016 at 11:25 PM, Bernd Walter = > wrote: >> On Thu, Mar 24, 2016 at 04:29:26PM +0100, Sylvain Garrigues wrote: >> > Hello, >> > >> > I have written a small (ugly) patch to be able to boot a kernel = directly without the ubldr loader while still using an external DTB = (Linux-style booting may pass the DTB location pointer in the r2 = register). >> > The patch is here: = https://reviews.freebsd.org/differential/diff/14577/ = >> > >> > I tested my patch successfully with the QEMU emulator with the -dtb = option available in recent versions, using a VERSATILEPB kernel without = FDT_STATIC. >> > # qemu-system-arm -M versatilepb -m 128M -kernel versatile.flash = -cpu arm1176 -dtb versatilepb.dtb >> > FYI, once the kernel is built, here is the script to build the = versatile.flash (adapted from gonzo, no longer need to clear the r0-r3 = registers): https://reviews.freebsd.org/P92 = > >> > >> > I also tested successfully my patch on my Raspberry Pi 2 using = U-BOOT and the RPI2 kernel with my patch applied (and the LINUX_BOOT_ABI = option): >> > u-boot> fatload mmc 0 0x200000 kernel.bin >> > u-boot> go 0x200000 >> > >> > That works. >> > >> > So now I thought I could even bypass u-boot and launch kernel.bin = directly from the Pi firmware??? But it doesn???t work, I don???t = understand why, and that is why I am writing here. >> > Here is the config.txt which I thought would work: >> > >> > kernel=3Dkernel.bin (instead of u-boot.bin) >> > kernel_address=3D0x200000 (line added because a FreeBSD kernel = needs to be loaded on a 1MB or 2MB boundary) >> > device_tree=3Drpi2.dtb >> > device_tree_address=3D0x100 >> > disable_commandline_tags=3D1 >> > >> > What am I missing? Is it even possible to boot kernel.bin directly = on the Pi (with my patch)? I found this static minimalist loader from = Andrew here: = https://github.com/freebsd/freebsd/commit/074d37d46c3f9b282cd2d849d997b1b3= 9acd710c = > - does it mean such a loader is necessary before the kernel? >> > >> > Thanks, >> > Sylvain >> > >> > PS: I know the =C2=AB official and supported =C2=BB way of booting = FreeBSD on the Pi is the u-boot + ubldr combination. I just would like = to finish this experiment and understand why kernel.bin cannot be booted = directly. >>=20 >> I don't know if it is part of ubldr, or if there is any additional = code, >> which runs before ubldr, but the Pis won't boot with the ARM CPU. >> Somewhere in the boot path is GPU bootcode, which then enables the = ARM >> CPU. >> You also need low level HW init, such as setting up the clocks and = RAM, >> which is not done in the Kernel. >>=20 >> What you are missing it the hardware init code that's in u-boot.bin. = This code sets up >> the board after the on-board GPU loads it into the ARM's address = space. That code >> sets up the ARM side of the world and hands off the code to ubldr = which finds the dtb >> and does the normal /boot/loader things as well (loading modules, = setting tunables >> and the like) then hands off to the kernel. You can eliminate ubldr = without a huge >> amount of effort, as you've found. However, eliminating u-boot.bin is = going to be >> a lot more work/ >>=20 >> tl;dr: The interface between u-boot.bin and ubldr/kernel.bin is quite = a bit different >> than between the initial bootstrap and u-boot.bin. So you can't just = drop in kernel.bin >> and have it work without replicating that interface. >>=20 >> Warner