From owner-freebsd-arm@freebsd.org Mon Mar 29 01:50:13 2021 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B43FC5B7DFA for ; Mon, 29 Mar 2021 01:50:13 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7wWS4y1vz3wLM for ; Mon, 29 Mar 2021 01:50:12 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Received: by mail-wr1-x429.google.com with SMTP id j7so11209914wrd.1 for ; Sun, 28 Mar 2021 18:50:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=TOjXwbaqhwHuKzxEqmfXdCZ26kIkLdhFa/85z5B2iDU=; b=tGRXPSU5qQ9L9pGT40pSAbvSbE8yNoq3gMby6oC4qvu/mg3WcWfuQHh31EqPAo1htd 2s4XgfcOUqd1MJx2SCENVPNZj1lAM0ETsD8jf0K0juthF6ENRGtyeS+U2D89+F1nO8co hO/xe1KXVj+UqwirC0ilvSnrTRGrWo/Pc8zBSuqU6a011gQJMz/Ksa+00WgXfjgED+6E E/vtjEozT+VNpox6GL9ineGbhntVQXnjqgD2IosnBfiXl9ZZygPxS40D0QzFXdVxop8P zIjpKZaKa7ucUhUWgsJxQGijKpla8nEtclN7NmkzP+IJAF118X0/RfJn0runejLB6DxR lQgQ== X-Gm-Message-State: AOAM530ulmmbqDf1rxMuGXxZfkuG1/hg+1O1iA8NrhKLq2WQPGFgxmrp cp7QMWL/pWyhx/QGlUnRtKI= X-Google-Smtp-Source: ABdhPJz/Vjn1UrPlZ+n3713e1Nept+29DoDf+9Uwj4IcYTkWIsTs+iLqVMw1+GYwmE/OQZsWFTSgvg== X-Received: by 2002:adf:dc4e:: with SMTP id m14mr26814024wrj.248.1616982611144; Sun, 28 Mar 2021 18:50:11 -0700 (PDT) Received: from [192.168.1.167] (dynamic-046-114-157-239.46.114.pool.telefonica.de. [46.114.157.239]) by smtp.googlemail.com with ESMTPSA id e17sm29868853wra.65.2021.03.28.18.50.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Mar 2021 18:50:10 -0700 (PDT) From: =?utf-8?Q?Klaus_K=C3=BCchemann?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: Any good alternative to Raspberry for Arm64? Date: Mon, 29 Mar 2021 03:50:08 +0200 References: <7b284f7718556f1cf0a7a205c98db6b1@pyret.net> <8F8F3491-3E1F-45C8-BF61-09F7557F48A5@googlemail.com> <265dbf9c33f4ce09c702c9d7fae93c9b@pyret.net> To: =?utf-8?Q?S=C3=B8ren_Schmidt?= , freebsd-arm@freebsd.org In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4F7wWS4y1vz3wLM X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[googlemail.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[googlemail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; NEURAL_HAM_SHORT(-0.65)[-0.648]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::429:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[googlemail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[46.114.157.239:received]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::429:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::429:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Porting FreeBSD to ARM processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 01:50:13 -0000 > Am 28.03.2021 um 22:02 schrieb S=C3=B8ren Schmidt = : >=20 >=20 > On 28 Mar 2021, at 21.48, Klaus K=C3=BCchemann = wrote: >>>=20 >>> Well, at least u-boot2021.01 has all the needed support to boot a = rockpro64 or pinebookpro from nvme/sata/usb/sd/mmc/net. >>> If you put u-boot into the SPI-flash it will boot from any of the = above media. >>>=20 >>> -- >>> S=C3=B8ren Schmidt >>> sos@deepcore.dk / sos@freebsd.org >>> "So much code to hack, so little time=E2=80=9C >>>=20 >>=20 >> yes, I used 2021.04-rcXX( master, self-compiled)=E2=80=A6 the strange = thing on Rock960-board was that it needed some reboots=20 >> until nvme was scanned correctly in u-boot-prompt. >> Thean it boots FreeBSD straight up and then hangs on mountroot> ( = because there=E2=80=99s no /dev/nda0 or /dev/nvmeXX). >> I presume an electrical issue since it was once(only once) detected = by adding nvme_load=3D=E2=80=9EYES=E2=80=9C& nvd_load=3D=E2=80=9EYES=E2=80= =9C in rc.conf.=20 >> The u-boot patch related to nvme on that board also was based on a = voltage-change . >> =E2=80=A6 I`ll post some debug-output the next hours=E2=80=A6 (`guess = pcie has mem alloc issue from dmesg)=E2=80=A6 >>=20 >> Anyone successfully booting from nvme on any aarch64-board ?? >=20 >=20 > Yes, both my rockpro64=E2=80=99s and my pinebookpro boots of NVMe just = fine, there is one caveat though, for it to work it needs to be a NVMe = device that uses as little power as possible (I use older Intel 660 IIRC = uses only 3.5W max), otherwise the HW cant cope.. >=20 >=20 > S=C3=B8ren Schmidt > sos@deepcore.dk / sos@freebsd.org > "So much code to hack, so little time=E2=80=9C >=20 > On 28 Mar 2021, at 21.48, Klaus K=C3=BCchemann = wrote: >>>=20 >> =E2=80=A6 I`ll post some debug-output the next hours=E2=80=A6 (`guess = pcie has mem alloc issue from dmesg)=E2=80=A6 This is one of the next hours :-). : with uSD plugged in(fbsd installed on it ) : rock960 =3D> pci rockchip_pcie pcie@f8000000: PCIe link training gen1 timeout! No such bus rock960 =3D> boot (from uSD)... --dmesg :--- ... pcib0: mem = 0xf8000000-0xf9ffffff,0xfd000000-0xfdffffff irq 6,7,8 on ofwbus0 pcib0: Gen1 link training timeouted: 0x00180001. pci0: on pcib0 pcib1: at device 0.0 on pci0 pcib0: failed to reserve resource for pcib1 pcib1: failed to allocate initial memory window: 0-0xfffff pci1: on pcib1 ... --- root@generic:~ # cat /boot/loader.conf ... nvme_load=3D"YES" nvd_load=3D"YES" .. root@generic:~ # reboot ( !!!)=20 -- U-Boot TPL 2021.04-rc4 (Mar 28 2021 - 22:44:49) Channel 0: LPDDR3, 800MHz BW=3D32 Col=3D10 Bk=3D8 CS0 Row=3D15 CS1 Row=3D15 CS=3D2 Die BW=3D16 = Size=3D2048MB Channel 1: LPDDR3, 800MHz BW=3D32 Col=3D10 Bk=3D8 CS0 Row=3D15 CS1 Row=3D15 CS=3D2 Die BW=3D16 = Size=3D2048MB 256B stride Trying to boot from BOOTROM Returning to boot ROM... U-Boot SPL 2021.04-rc4 (Mar 28 2021 - 22:44:49 +0000) Trying to boot from MMC2 NOTICE: BL31: v2.4(release): NOTICE: BL31: Built : 22:59:19, Jan 30 2021 U-Boot 2021.04-rc4 (Mar 28 2021 - 22:44:49 +0000) SoC: Rockchip rk3399 Reset cause: RST Model: 96boards Rock960 DRAM: 3.9 GiB PMIC: RK808=20 MMC: mmc@fe310000: 2, mmc@fe320000: 1, sdhci@fe330000: 0 Loading Environment from MMC... *** Warning - bad CRC, using default = environment In: serial Out: serial Err: serial Model: 96boards Rock960 starting USB...net found. Bus usb@fe380000: USB EHCI 1.00 Bus usb@fe3a0000: USB OHCI 1.0 Bus usb@fe3c0000: USB EHCI 1.00 Bus usb@fe3e0000: USB OHCI 1.0 Bus dwc3: Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 Bus dwc3: Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 scanning bus usb@fe380000 for devices... 1 USB Device(s) found scanning bus usb@fe3a0000 for devices... 1 USB Device(s) found scanning bus usb@fe3c0000 for devices... 1 USB Device(s) found scanning bus usb@fe3e0000 for devices... 1 USB Device(s) found scanning bus dwc3 for devices... 1 USB Device(s) found scanning bus dwc3 for devices... 1 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found Hit any key to stop autoboot: 0=20 rock960 =3D> pci Scanning PCI devices on bus 0 BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x1d87 0x0100 Bridge device 0x04 rock960 =3D> nvme scan rock960 =3D> nvme info Device 0: Vendor: 0x15b7 Rev: 211070WD Prod: 20251C807274 =20 Type: Hard Disk Capacity: 238475.1 MB =3D 232.8 GB (488397168 x 512) =E2=80=94=E2=80=94 Now I swap out the uSD while device-power is on ( so that only the nvme = has freebsd installed) ( u-boot is on the eMMC !!) : rock960 =3D> run nvme_boot =E2=80=A6.. =E2=80=A6. OK lsdev=20 disk devices: disk0: 249737216 X 512 blocks (removable) efipart_readwrite: rw=3D1, blk=3D0 size=3D1 status=3D7 disk1: 59768832 X 512 blocks (removable) disk1p1: Unknown disk1p2: Unknown disk1p3: Unknown disk1p4: EFI disk1p5: Unknown disk2: 488397168 X 512 blocks ( this is the nvme-drive !!!) disk2s1: DOS/Windows disk2s2: FreeBSD disk2s2a: FreeBSD UFS http: (unknown) net devices: - OK show currdev disk2p2: -- OK boot disk2p2:/boot/kernel/kernel .... !!!!!!!Definitely boots from nvme !!!!! .... Bla bla bla ..... pcib0: mem = 0xf8000000-0xf9ffffff,0xfd000000-0xfdffffff irq 6,7,8 on ofwbus0 pcib0: Gen1 link training timeouted: 0x00180001. pci0: on pcib0 pcib1: at device 0.0 on pci0 pcib0: failed to reserve resource for pcib1 pcib1: failed to allocate initial memory window: 0-0xfffff pci1: on pcib1 .......... mountroot: waiting for device /dev/ufs/rootfs... Mounting from ufs:/dev/ufs/rootfs failed with error 19. ... mountroot> ? ... shows only partitions of the eMMC ... = FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20210318-a771bf748f9-245511.img.xz = is dd`d to the nvme-drive .. previously cleanly new formatted with = dd(/dev/zero) & gpart ... not sure to post an freebsd-bug-report because it could be u-boot or = hardware-related like S=C3=B8ren Schmidt described : <<(I use older = Intel 660 IIRC uses only 3.5W max), otherwise the HW cant cope..>> Drive is a WD-blue nvme (M-key), directly plugged into the M.2-slot of = the Rock960 , Doesn't matter whether 12V/2a (default plug) or 12V/3a is used . The real strange thing is that it definitely boots up from nvme. Original u-boot-patch is : https://lists.denx.de/pipermail/u-boot/2020-April/408490.html Thanks for reading=20 K.