Date: Sun, 29 Sep 2019 23:07:10 -0400 From: Dennis Clarke <dclarke@blastwave.org> To: freebsd-riscv@freebsd.org Subject: Re: QEMU 4.1 and RISC-V Message-ID: <1bc10ee3-b717-61a9-2e5d-7fb8538234f2@blastwave.org> In-Reply-To: <CAKBkRUw4Ebawyt5GtGdM6wKCxLuX54f8cR_cQK13LKx1_CpHYw@mail.gmail.com> References: <CAKBkRUw%2BgGKOzU1_JWv_FUx5A9jBnuq0MvQh15Yy9NZ0PKYR9A@mail.gmail.com> <CADeAsy30gdZJa9S7easL6tTry4RrYzDBazx2u2cLtF8TrE9qMw@mail.gmail.com> <CAKBkRUz_V4Cz%2BHSxsazWJ-i4-J6%2B2qxfHZUgUcJi-bfoEhsSxg@mail.gmail.com> <CADeAsy0uiLr7=8f9eSBJn-wCSTiZfXr6Ua37Ch7ffTwkSvegqw@mail.gmail.com> <CAKBkRUxRW5v=FfuziXrrEHrz_ui4MjXrOOf32376HxVTH0bYDQ@mail.gmail.com> <CAKBkRUw4Ebawyt5GtGdM6wKCxLuX54f8cR_cQK13LKx1_CpHYw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/29/19 10:54 PM, Li-Wen Hsu wrote: > On Mon, Sep 30, 2019 at 4:28 AM Li-Wen Hsu <lwhsu@freebsd.org> wrote: >> >> On Mon, Sep 30, 2019 at 4:12 AM Mitchell Horne <mhorne@freebsd.org> wrote: >>> >>> On Sun, Sep 29, 2019 at 4:56 PM Li-Wen Hsu <lwhsu@freebsd.org> wrote: >>>> >>>> On Sun, Sep 29, 2019 at 10:48 PM Mitchell Horne <mhorne@freebsd.org> wrote: >>>>> >>>>> >>>>> On Sun, Sep 29, 2019, 15:17 Li-Wen Hsu <lwhsu@freebsd.org> wrote: >>>>>> >>>>>> I found that RISC-V test started failing recently: >>>>>> https://ci.freebsd.org/job/FreeBSD-head-riscv64-test/ >>>>>> >>>>>> After some checking, I found the issue is our bbl image cannot boot >>>>>> with this command: >>>>>> >>>>>> qemu-system-riscv64 -nographic -m 2048M -kernel ./bbl >>>>>> >>>>>> in QEMU 4.1, while 3.1 and 4.0 are fine. I've switched the QEMU >>>>>> version used in that job to 4.0, but I think it would be best if we >>>>>> can find the root cause is in FreeBSD or QEMU. >>>>>> >>>>>> Can anyone also help on this? >>>>> >>>>> >>>>> Hi Li-Wen, >>>>> >>>>> I can take a look in the coming week if you're okay with the workaround for the time being. Locally >>>>> I've had no issue running bbl with qemu 4.1 but perhaps there's something more specific that the >>>>> CI script is doing. >>>>> >>>>> I'm hoping we will soon be able to replace bbl with the more actively maintained OpenSBI firmware. >>>>> I have some pending changes that will allow us to boot FreeBSD with it, and after that I'd like to >>>>> create ports for both bootloaders, which I hope will make the CI setup a little simpler and less fragile >>>>> than building bbl currently is. >>>> >>>> Thanks very much, Mitchell, >>>> >>>> Let's narrow down the scope, are you using the qemu-devel pkg from >>>> official repository or building by yourself? >>>> >>> >>> This was using qemu that I had built myself, but the executable from qemu-devel >>> works for me as well. >>> >>>> And can you try this bbl image? >>>> https://artifact.ci.freebsd.org/snapshot/head/r352870/riscv/riscv64/bbl.xz >>> >>> I had no issue booting this image with my self-built qemu or the one from >>> pkg. The only observable difference from qemu 3.1 is that a warning is >>> printed to stderr, but considering the CI script is timing out I don't think >>> that is the issue. >> >> What's your command line to launch qemu? The CI (and I) uses this: >> >> qemu-system-riscv64 -nographic -m 2048M -kernel ./bbl >> >> I also tried to build qemu locally, and result is the same. I also did >> a git bisect: >> >> 0ac24d56c5e7d32423ea78ac58a06b444d1df04d is the first bad commit > > Hmm, after checking with wiki.freebsd.org/riscv, I found that we need > to specify `-machine virt` now. > > QEMU 4.1 default machine for riscv64 is "spike" not "spike_v1.10" > anymore, and need other more arguments. This works for me but I would switch over to "spike" if that is the forward looking plan : styx$ styx$ ls -l bbl rv64imafdc_disk0.img -rw-r--r-- 1 riscv riscv 10660264 Aug 28 00:31 bbl -rw-r--r-- 1 riscv riscv 34359738368 Sep 11 06:54 rv64imafdc_disk0.img styx$ qemu-system-riscv64 -nographic -machine virt \ > -smp 2 -m 4G \ > -kernel /home/riscv/qemu/bbl \ > -object rng-random,filename=/dev/urandom,id=rng0 \ > -device virtio-rng-device,rng=rng0 \ > -drive file=/home/riscv/qemu/rv64imafdc_disk0.img,format=raw,index=0,media=disk,id=hd0 \ > -device virtio-blk-device,drive=hd0 \ > -device virtio-net-device,netdev=usernet \ > -netdev user,id=usernet,hostfwd=tcp::10000-:22 bbl loader ---<<BOOT>>--- KDB: debugger backends: ddb KDB: current backend: ddb Found 2 CPUs in the device tree Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT r350568 QEMU riscv gcc version 8.1.0 (FreeBSD Ports Collection for riscv64) WARNING: WITNESS option enabled, expect reduced performance. Preloaded elf64 kernel "kernel" at 0xffffffc0007e1cc0. CPU(0): Unknown Implementer Unknown Processor real memory = 4293918720 (4095 MB) Physical memory chunk(s): 0x0000000080a42000 - 0x0000000080bfffff, 1826816 bytes (446 pages) 0x0000000080d00000 - 0x00000001797c5fff, 4172046336 bytes (1018566 pages) avail memory = 4156452864 (3963 MB) Starting CPU 1 (hart 0) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. VIMAGE (virtualized network stack) enabled ULE: setup cpu 0 ULE: setup cpu 1 random: entropy device external interface nfslock: pseudo-device crypto: <crypto core> null: <full device, null device, zero device> mem: <memory> openfirm: <Open Firmware control device> ofwbus0: <Open Firmware Device Tree> simplebus0: <Flattened device tree simple bus> on ofwbus0 plic0: <RISC-V PLIC> mem 0xc000000-0xfffffff irq 9,10,11,12 on simplebus0 timer0: <RISC-V Timer> Timecounter "RISC-V Timecounter" frequency 10000000 Hz quality 1000 Event timer "RISC-V Eventtimer" frequency 10000000 Hz quality 1000 rcons0: <RISC-V console> ofwbus0: no default resources for rid = 0, type = 4 ofwbus0: no default resources for rid = 0, type = 4 ofwbus0: no default resources for rid = 0, type = 4 uart0: <16550 or compatible> mem 0x10000000-0x100000ff irq 0 on ofwbus0 uart0: console (76800,n,8,1) uart0: fast interrupt uart0: PPS capture mode: DCD ofwbus0: <test@100000> mem 0x100000-0x100fff compat sifive,test0 (no driver attached) virtio_mmio0: <VirtIO MMIO adapter> mem 0x10008000-0x10008fff irq 1 on ofwbus0 virtio_mmio1: <VirtIO MMIO adapter> mem 0x10007000-0x10007fff irq 2 on ofwbus0 vtblk0: <VirtIO Block Adapter> on virtio_mmio1 virtio_mmio1: host features: 0x31000ed4 <EventIdx,RingIndirect,NotifyOnEmpty,ConfigWCE,Topology,WriteCache,SCSICmds,BlockSize,DiskGeometry,MaxNumSegs> virtio_mmio1: negotiated features: 0x10000e54 <RingIndirect,ConfigWCE,Topology,WriteCache,BlockSize,DiskGeometry,MaxNumSegs> vtblk0: 32768MB (67108864 512 byte sectors) virtio_mmio2: <VirtIO MMIO adapter> mem 0x10006000-0x10006fff irq 3 on ofwbus0 vtnet0: <VirtIO Networking Adapter> on virtio_mmio2 virtio_mmio2: host features: 0x39bf8064 <EventIdx,RingIndirect,0x8000000,NotifyOnEmpty,SetMacAddress,GuestAnnounce,RxModeExtra,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxAllGSO,MacAddress,0x4> virtio_mmio2: negotiated features: 0x308f8060 <EventIdx,RingIndirect,SetMacAddress,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxAllGSO,MacAddress> vtnet0: bpf attached vtnet0: Ethernet address: 52:54:00:12:34:56 virtio_mmio3: <VirtIO MMIO adapter> mem 0x10005000-0x10005fff irq 4 on ofwbus0 virtio_mmio4: <VirtIO MMIO adapter> mem 0x10004000-0x10004fff irq 5 on ofwbus0 virtio_mmio5: <VirtIO MMIO adapter> mem 0x10003000-0x10003fff irq 6 on ofwbus0 virtio_mmio6: <VirtIO MMIO adapter> mem 0x10002000-0x10002fff irq 7 on ofwbus0 virtio_mmio7: <VirtIO MMIO adapter> mem 0x10001000-0x10001fff irq 8 on ofwbus0 cpulist0: <Open Firmware CPU Group> on ofwbus0 cpu0: <Open Firmware CPU> on cpulist0 cpu0: Nominal frequency 1000Mhz riscv64_cpu0: register <0> cpu1: <Open Firmware CPU> on cpulist0 cpu1: Nominal frequency 1000Mhz riscv64_cpu1: register <1> cryptosoft0: <software crypto> crypto: assign cryptosoft0 driver id 0, flags 0x6000000 crypto: cryptosoft0 registers alg 1 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 2 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 3 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 4 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 5 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 16 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 6 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 7 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 32 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 18 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 19 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 20 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 8 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 15 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 9 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 10 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 13 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 14 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 34 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 35 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 36 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 37 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 11 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 22 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 23 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 25 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 24 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 26 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 27 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 28 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 21 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 17 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 29 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 30 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 31 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 40 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 39 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 38 flags 0 maxoplen 0 Device configuration finished. procfs registered Timecounters tick every 10.000 msec vlan: initialized, using hash tables with chaining lo0: bpf attached tcp_init: net.inet.tcp.tcbhashsize auto tuned to 32768 IPsec: Initialized Security Association Processing. GEOM: new disk vtbd0 Trying to mount root from ufs:/dev/vtbd0 []... Release APs CPU(1): Unknown Implementer Unknown Processor WARNING: WITNESS option enabled, expect reduced performance. Warning: no time-of-day clock registered, system time will not be set accurately start_init: trying /sbin/init . . . -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1bc10ee3-b717-61a9-2e5d-7fb8538234f2>