From owner-freebsd-arm@freebsd.org Fri Mar 25 21:35:53 2016 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0F6DADEB13 for ; Fri, 25 Mar 2016 21:35:53 +0000 (UTC) (envelope-from sylgar@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E971610B9 for ; Fri, 25 Mar 2016 21:35:52 +0000 (UTC) (envelope-from sylgar@gmail.com) Received: by mail-wm0-x234.google.com with SMTP id l68so34990127wml.1 for ; Fri, 25 Mar 2016 14:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=86A0mDRygApykRcebwqD6OKC1FS2gXUJag+SxIy1gMI=; b=dWLO/DnCzIBvTSUcVdug/3XYy4R/Cku+TEXcT75dg3jJLuC8Kh8xGT3rPQR6FYwL0v 94NQvooONuFgRlrGyXmHvDIgC3Ip6/Rd2/oxA4JKlgAPHuQhwCtqNnJKaZwHKpQROS8n BfkY2K89aBq7aloR6J3+lub1OD0uTXk1wJ6/AhdFz7TTphKunMfRW+/W4Kl4CcPM824K id8tlobu6VN53+RzkjNqJp8rrvitNhLZXIXzBL22n36BkYqnvqRnl6Rnq2PB9eCqr5p/ 5Wm0/UZYETAtf/374SiP5oehkCtTax59HBCmBqqau9Ygck0hkQpebseZPhwRm7XCB4pJ 5l7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=86A0mDRygApykRcebwqD6OKC1FS2gXUJag+SxIy1gMI=; b=D4DoJx7xotpiYhhCoZk6DwC81NmQsMo4D0nz1xCW/SFM5IC9wCF2DzrdsxZOzRLB/1 hI6iygG4e4bKLuiQlLA8t0yrxvVHxUf/Pnuja7RvSCSTZ4T9Jq5pIFRYeeJJeJGCCmG5 gFwUkz9lHfjernDCqEXIcsGklB9x+sLWPf/R8uhj9Sz3q90IhPF0obveUk2BI8VYyUsk mSvsIPC01WGIV9h2GFIJR4tZi/MLc+kIzhvlwLTOyP9EVzYH0/FABnB5+cf3Q0S6BTS6 /rnAvX6SZyacyP5vqG0pLIascgUBkdyC0Op+tLxxetbQQHSwF6/D5RKAqI271+4X4dvK APfw== X-Gm-Message-State: AD7BkJKThSHOuc1gEg8Sb6qrj/7cDV06mgjJT2uzNF4AAvogd3aQDsoIvVN36lgeQuSl6g== X-Received: by 10.28.109.4 with SMTP id i4mr481846wmc.44.1458941750104; Fri, 25 Mar 2016 14:35:50 -0700 (PDT) Received: from [192.168.0.4] ([81.56.235.196]) by smtp.gmail.com with ESMTPSA id up6sm13593604wjc.6.2016.03.25.14.35.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 25 Mar 2016 14:35:48 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Booting kernel.bin directly on Raspberry Pi / external DTB support From: Sylvain Garrigues In-Reply-To: Date: Fri, 25 Mar 2016 22:35:47 +0100 Cc: ticso@cicely.de, freebsd-arm Message-Id: References: <1CCA59DC-5539-4CFB-81BA-0112E2120B3B@gmail.com> <20160325052509.GE48704@cicely7.cicely.de> <97BB142C-D335-469B-9514-D1210E869C5D@gmail.com> <2A98ACB7-2815-4169-951A-0123C71D9987@gmail.com> To: Warner Losh X-Mailer: Apple Mail (2.3124) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Mar 2016 21:35:53 -0000 > Le 25 mars 2016 =C3=A0 19:18, Warner Losh a =C3=A9crit = : >=20 > Have you looked at NetBSD to see what they do? It sounds like the SD = driver in the boot roms is much faster than the one in u-boot.bin. = Perhaps there's some init code we could import? Also, what are the = kernel sizes? Maybe their kernel is smaller than ours for some reason = and we should look at why... NetBSD=E2=80=99s kernel size for RPI2 is around 5.5MB which is similar = to our kernel. It is booted directly by the firmware and no specific config.txt is = required. So it looks exactly like a linux kernel, it even has a = cmdline.txt file which allows the passing of boot parameters (root=3Dld0a = console=3Dfb) You mentioned the ARM side of the RPI needs to be set up before being = able to run the FreeBSD kernel: for my understanding, any idea about = what the FreeBSD kernel requires exactly (apart from the metadata and = all the module loading stuff)? I mean what does it need in terms of = board set up - i.e. in other words what does u-boot does that is = required by the FreeBSD kernel? I was under the understanding that the = firmware sets up everything and even patches the DTB with board specific = values (memory size, etc). Given that the firmware passes the patched = DTB to the binary it loads, I expected the kernel.bin to boot directly, = but I am obviously wrong and lack some embedded knowledge.=20 I am sorry to ask here, I guess my questions aren=E2=80=99t of interest = to many people, but I want to learn more and can=E2=80=99t find = documentation on this matter. =20 Cheers, Sylvain.=20 > On Fri, Mar 25, 2016 at 12:14 PM, Sylvain Garrigues > wrote: > u-boot> fatload mmc 0 0x100000 kernel.bin >=20 > takes around the same time than ubldr (actually slightly less, let=E2=80= =99s say 4s - but that=E2=80=99s still so slow...) >=20 > Happy to help or test anything to get FreeBSD=E2=80=99s kernel first = line appear on the screen near =C2=AB instantaneously =C2=BB (~ 1s) like = our NetBSD or Linux friends. >=20 > Sylvain >=20 >=20 >> Le 25 mars 2016 =C3=A0 19:01, Warner Losh > a =C3=A9crit : >>=20 >>=20 >>=20 >> On Fri, Mar 25, 2016 at 11:55 AM, Sylvain Garrigues > wrote: >> Thank you.=20 >>=20 >> I wanted to boot the kernel directly so as to: >> 1/ remove the dependency on a 3rd party loader >> 2/ reduce boot time and make it comparable with NetBSD and Linux = (which don=E2=80=99t use u-boot) >>=20 >> NetBSD and Linux on the Pi boot so much faster than our FreeBSD = image: less than 1 second after power-up, the first kernel copyright = line appears on my LCD. For the FreeBSD images, the longest part is = ubldr loading the kernel. It takes around 4/5 seconds (with the same SD = card). >>=20 >> I used recent u-boot versions (the current u-boot-rpi2 port is = getting old and is not in sync with the u-boot-rpi port recently = updated), tried with dcache enabled, but the boot process still takes = several seconds, most of the time being spent in ubldr loading the = kernel. >>=20 >> I have no idea how we could improve the booting time? I wanted to = experiment with booting the kernel directly as the first ARM program on = my Pi, but as you explained I need some init code. I managed to compile = Andrew=E2=80=99s one = (https://github.com/freebsd/freebsd/commit/074d37d46c3f9b282cd2d849d997b1b= 39acd710c = ) but I don=E2=80=99t know how to use it. Andrew, if you read = this ;-) >>=20 >> I wonder if the loader could be more efficient at loading off the SD = card. The SD cards can do 10-20MB/s easily, and the ARM kernel is around = 5MB last time I checked. 5 seconds is 1MB/s, which is much slower than = we know the hardware can do. I don't know if that's because the callback = mechanism in u-boot.bin is so slow (that's what ubldr uses to load the = kernel), or there's something inherently slow about our loader. Some = analysis here might be quite useful. I'm guessing that we're not getting = all the benefits from streaming read mode because we're doing I/Os that = are tiny for some reason. But I haven't looked to make sure. >>=20 >> What's the speed when booting directly from u-boot.bin? >>=20 >> Warner >>=20 >>=20 >>=20 >> =20 >>> Le 25 mars 2016 =C3=A0 06:48, Warner Losh > a =C3=A9crit : >>>=20 >>>=20 >>>=20 >>> On Thu, Mar 24, 2016 at 11:25 PM, Bernd Walter = > wrote: >>> On Thu, Mar 24, 2016 at 04:29:26PM +0100, Sylvain Garrigues wrote: >>> > Hello, >>> > >>> > I have written a small (ugly) patch to be able to boot a kernel = directly without the ubldr loader while still using an external DTB = (Linux-style booting may pass the DTB location pointer in the r2 = register). >>> > The patch is here: = https://reviews.freebsd.org/differential/diff/14577/ = >>> > >>> > I tested my patch successfully with the QEMU emulator with the = -dtb option available in recent versions, using a VERSATILEPB kernel = without FDT_STATIC. >>> > # qemu-system-arm -M versatilepb -m 128M -kernel versatile.flash = -cpu arm1176 -dtb versatilepb.dtb >>> > FYI, once the kernel is built, here is the script to build the = versatile.flash (adapted from gonzo, no longer need to clear the r0-r3 = registers): https://reviews.freebsd.org/P92 = > >>> > >>> > I also tested successfully my patch on my Raspberry Pi 2 using = U-BOOT and the RPI2 kernel with my patch applied (and the LINUX_BOOT_ABI = option): >>> > u-boot> fatload mmc 0 0x200000 kernel.bin >>> > u-boot> go 0x200000 >>> > >>> > That works. >>> > >>> > So now I thought I could even bypass u-boot and launch kernel.bin = directly from the Pi firmware??? But it doesn???t work, I don???t = understand why, and that is why I am writing here. >>> > Here is the config.txt which I thought would work: >>> > >>> > kernel=3Dkernel.bin (instead of u-boot.bin) >>> > kernel_address=3D0x200000 (line added because a FreeBSD kernel = needs to be loaded on a 1MB or 2MB boundary) >>> > device_tree=3Drpi2.dtb >>> > device_tree_address=3D0x100 >>> > disable_commandline_tags=3D1 >>> > >>> > What am I missing? Is it even possible to boot kernel.bin directly = on the Pi (with my patch)? I found this static minimalist loader from = Andrew here: = https://github.com/freebsd/freebsd/commit/074d37d46c3f9b282cd2d849d997b1b3= 9acd710c = > - does it mean such a loader is necessary before the kernel? >>> > >>> > Thanks, >>> > Sylvain >>> > >>> > PS: I know the =C2=AB official and supported =C2=BB way of booting = FreeBSD on the Pi is the u-boot + ubldr combination. I just would like = to finish this experiment and understand why kernel.bin cannot be booted = directly. >>>=20 >>> I don't know if it is part of ubldr, or if there is any additional = code, >>> which runs before ubldr, but the Pis won't boot with the ARM CPU. >>> Somewhere in the boot path is GPU bootcode, which then enables the = ARM >>> CPU. >>> You also need low level HW init, such as setting up the clocks and = RAM, >>> which is not done in the Kernel. >>>=20 >>> What you are missing it the hardware init code that's in u-boot.bin. = This code sets up >>> the board after the on-board GPU loads it into the ARM's address = space. That code >>> sets up the ARM side of the world and hands off the code to ubldr = which finds the dtb >>> and does the normal /boot/loader things as well (loading modules, = setting tunables >>> and the like) then hands off to the kernel. You can eliminate ubldr = without a huge >>> amount of effort, as you've found. However, eliminating u-boot.bin = is going to be >>> a lot more work/ >>>=20 >>> tl;dr: The interface between u-boot.bin and ubldr/kernel.bin is = quite a bit different >>> than between the initial bootstrap and u-boot.bin. So you can't just = drop in kernel.bin >>> and have it work without replicating that interface. >>>=20 >>> Warner >=20 >=20