From owner-freebsd-arm@freebsd.org Mon Jun 29 00:21:38 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C610359001 for ; Mon, 29 Jun 2020 00:21:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49w7TD3nzGz42H6 for ; Mon, 29 Jun 2020 00:21:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: TIyW5w4VM1mDlP9.riimGDLBkkJiCWoa1Sn_.Jsrnmsxw9pY1yS.Bu2id407SnF QYgmxYlNH_YxH.mAPkXMaxxYyj9N.WufaoMQxWuOKXVa8uBvqa7iDwfTylXO1Gux1N9.Q2dvTcii SwG8RpQ93ge7VGaHwueqRFAEVos1j3LK0LF.kvZk7B7gSbXCyHIzWmTQtuB2hnuj39YyTKaiDufH d0TEL_ZUqbCk6PXQopHcCFXJ8MI8CW.5gYpWww5NLaztxfzm6EFToqVdRttFo2sdKzK3xX7KmVdH KRS87a__A1AYvcuR0Qq.2ReGiV.Ppo4g.k6Wp6kYCrHE_di9R7l4T7UmgFbAbrk67UPmv2o3EQOg m_kBjr39GWpTcOz6BXbCcQheemkbyF6xMj09F0uNwU1mGHsXCQDa4t7qjYYLoA1ZILTMqOMnNL5q LXjlsBX2DKPtTrzcQ04L8YYXPgl6130vjaCCXOoYE01pGGRmXblPmxlZ5ovew3OBbGbIlnoivE8K 3Yq.4MrF9FNIJtjm2gjU42eHg3nQ2bPfKWGCVksqOQxjsXN93pimxWau2FgwvV26KHdkPIc177rF 7jv0dQb0ARYQwTQfgVk7.2m2N9IdKD60e5A7dsAHi7N6gwlpkANSAY4KpGJquZacmPrQIkzeKK0b ELW_AH7mRo.noLPLYuHDV7N_O02jOURyx_Q1J7C3g968h.ki163b4altVbtyvz2Yhz1whrBGppIu 1x2dorVVdN0A5nYCacMXLCjZO7jGNoyLZwop9UHN4fR4SzuYbDpVj3yuxrFOVcA1KGgnTv9LFUmD WKnMHVDfO4KqW.YbBimyKfjZ0dIWeR_wkB1z1Hb3BeWjloj8mI6PNiBHtd0SxO1029lk6tZp8BpG 01QVYcnhO5QN4rsBTw9sm6kYNhtyRiY7EvnGZOv7eHON6.uwtXMdl0gg9VbchItQ1hagBUUVT1gy 1BVGgoAuEm6g4eAtcSzzVLg2BN2uYsbsgidZaAPwgKFOOuLGQPjAOK_B7UzA89_xXOFgIhk.aGOu UBoYJA0RNgkeVW6obQRmPyEFCTg292U6q1yhcED5BJA7e18h9K83wssg295vwo6rWRGBw8I6vfJY FPBEIaLYU8SRjGnB7hW.cvr2RwYNti1EkxZaC3kssgG5n4dYajDQl6kZrUS1sBtGrlEq3ypL97Oe rzSkxYmfvAw02nGOhiWyT87bQutdQ4JTpx3pBUH6xVv3IEoDMBFklk3clxgN5zSgnq8kcoBzYpcT s6StEEe8bAdB.3BvOmZCA6n_AF3bEROhWMHqay9QyVPdorpA4nJg0oM3lZgi.bt2L_Yp45JtLplF w3z4KWfMurIoudgXpTIQDGm6gANx_XB5aeJPnF2jtWPmi8LGNLhjEGYDroOgwntzLztupbT1ddCd R2Sw0q9lwzM_FqmUIsuWGX_oPvpNmUsUDkgs1aucQOJm6LClQzXSnLh3.5mmw Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Mon, 29 Jun 2020 00:21:34 +0000 Received: by smtp418.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 885ce9d5ecd02d64230782c7cc6cecd1; Mon, 29 Jun 2020 00:21:34 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: panic: non-current pmap on RPI3 on CURRENT (GENERIC) #4 r356366 From: Mark Millard In-Reply-To: <20200628195043.GA10909@www.zefox.net> Date: Sun, 28 Jun 2020 17:21:33 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <875C12B8-1E71-4808-AD3E-1B44D0AD9AF4@yahoo.com> References: <20200108235630.GA17485@www.zefox.net> <20200109115123.GZ23031@kib.kiev.ua> <20200109172314.GA20008@www.zefox.net> <20200628195043.GA10909@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 49w7TD3nzGz42H6 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.29 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-0.96)[-0.956]; NEURAL_HAM_MEDIUM(-0.96)[-0.962]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; NEURAL_HAM_SHORT(-0.87)[-0.874]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2020 00:21:38 -0000 On 2020-Jun-28, at 12:50, bob prohaska wrote: > On Thu, Jan 09, 2020 at 09:23:14AM -0800, bob prohaska wrote: >> On Thu, Jan 09, 2020 at 01:51:23PM +0200, Konstantin Belousov wrote: >>>=20 >>> It would be useful to see both the curcpu pc_curpmap content, >>> and dump both *(struct pmap *)0xfffffd000385f5a0 and *pc_curpmap >>> from the vmcore. >=20 > The Pi3 is now up to r362283 and just reported: >=20 > panic: non-current pmap 0xfffffd000142d440 > cpuid =3D 0 > time =3D 1593368952 > KDB: stack backtrace: > db_trace_self() at db_trace_self_wrapper+0x28 > pc =3D 0xffff00000075e24c lr =3D 0xffff00000010a468 > sp =3D 0xffff00005a86d2e0 fp =3D 0xffff00005a86d4e0 >=20 > db_trace_self_wrapper() at vpanic+0x194 > pc =3D 0xffff00000010a468 lr =3D 0xffff000000419dcc > sp =3D 0xffff00005a86d4f0 fp =3D 0xffff00005a86d540 >=20 > vpanic() at panic+0x44 > pc =3D 0xffff000000419dcc lr =3D 0xffff000000419b74 > sp =3D 0xffff00005a86d550 fp =3D 0xffff00005a86d600 >=20 > panic() at pmap_remove_pages+0x908 > pc =3D 0xffff000000419b74 lr =3D 0xffff000000776e00 > sp =3D 0xffff00005a86d610 fp =3D 0xffff00005a86d680 >=20 > pmap_remove_pages() at vmspace_exit+0x104 > pc =3D 0xffff000000776e00 lr =3D 0xffff0000006f7024 > sp =3D 0xffff00005a86d690 fp =3D 0xffff00005a86d6e0 >=20 > vmspace_exit() at exit1+0x48c > pc =3D 0xffff0000006f7024 lr =3D 0xffff0000003d13fc > sp =3D 0xffff00005a86d6f0 fp =3D 0xffff00005a86d750 >=20 > exit1() at sys_sys_exit+0x10 > pc =3D 0xffff0000003d13fc lr =3D 0xffff0000003d0f6c > sp =3D 0xffff00005a86d760 fp =3D 0xffff00005a86d7b0 >=20 > sys_sys_exit() at do_el0_sync+0x3f8 > pc =3D 0xffff0000003d0f6c lr =3D 0xffff00000077dac8 > sp =3D 0xffff00005a86d7c0 fp =3D 0xffff00005a86d830 >=20 > do_el0_sync() at handle_el0_sync+0x90 > pc =3D 0xffff00000077dac8 lr =3D 0xffff000000760a24 > sp =3D 0xffff00005a86d840 fp =3D 0xffff00005a86d980 >=20 > handle_el0_sync() at 0x404bd678 > pc =3D 0xffff000000760a24 lr =3D 0x00000000404bd678 > sp =3D 0xffff00005a86d990 fp =3D 0x0000ffffffffe960 >=20 > KDB: enter: panic > [ thread pid 42572 tid 100137 ] > Stopped at 0x4053fcfc > db>=20 >=20 >=20 > This time it was in the early stages of compiling www/chromium. > Boot and root are from a mechanical hard disk, the dying top page > was: >=20 > last pid: 42562; load averages: 1.40, 1.37, 1.38 = up 8+22:05:11 11:29:10 > 47 processes: 3 running, 44 sleeping > CPU: 27.1% user, 0.0% nice, 11.4% system, 0.4% interrupt, 61.0% idle > Mem: 92M Active, 237M Inact, 1468K Laundry, 158M Wired, 77M Buf, 415M = Free > Swap: 6042M Total, 194M Used, 5849M Free, 3% Inuse > packet_write_wait: Connection to 50.1.20.28 port 22: Broken pipe > bob@raspberrypi:~ $ R PRI NICE SIZE RES STATE C TIME WCPU = COMMAND > 42514 root 1 88 0 111M 63M CPU2 2 0:08 100.21% = c++ > 81775 bob 1 52 0 13M 352K wait 0 9:50 0.35% = sh > 29366 bob 1 20 0 14M 1340K CPU0 0 3:00 0.22% = top > 29351 bob 1 20 0 20M 936K select 2 0:15 0.03% = sshd > 639 root 1 20 0 13M 972K select 3 0:28 0.01% = syslogd > 30908 root 1 52 0 194M 40M select 1 1:52 0.00% = ninja > 46086 bob 1 20 0 20M 312K select 0 1:48 0.00% = sshd > ...... >=20 > I'll update OS sources and try again, if somebody can tell me > how to capture more useful information I'll try that. Do you have your system set up to allow it to dump to the swap/paging space for panics and then to put a copy in the /var/crash/ area during the next boot? Konstantin B. was asking for information from such a dump. Note: A dump can be requested at the db> prompt by typing a "dump" command at the prompt, if you have set up to have a dump target identified, usually a swap/paging partition. If it works, the next boot would take some time putting material into /var/crash. An example of such materials in /var/crash/ is (2 dumps): # ls -ldT /var/crash/* -rw-r--r-- 1 root wheel 2 Jun 16 19:58:17 2020 = /var/crash/bounds -rw-r--r-- 1 root wheel 32484 Jun 11 20:34:35 2020 = /var/crash/core.txt.3 -rw-r--r-- 1 root wheel 32498 Jun 16 19:58:47 2020 = /var/crash/core.txt.4 -rw------- 1 root wheel 561 Jun 11 20:34:04 2020 = /var/crash/info.3 -rw------- 1 root wheel 562 Jun 16 19:58:17 2020 = /var/crash/info.4 lrwxr-xr-x 1 root wheel 6 Jun 16 19:58:17 2020 = /var/crash/info.last -> info.4 -rw-r--r-- 1 root wheel 5 Feb 22 02:37:33 2016 = /var/crash/minfree -rw------- 1 root wheel 9424896 Jun 11 20:34:04 2020 = /var/crash/vmcore.3 -rw------- 1 root wheel 9424896 Jun 16 19:58:17 2020 = /var/crash/vmcore.4 lrwxr-xr-x 1 root wheel 8 Jun 16 19:58:17 2020 = /var/crash/vmcore.last -> vmcore.4 Do you have devel/gdb installed? It supplies a /usr/local/bin/kgdb for looking at such vmcore.* files. It is important that the kernel debug information still match the vmcore as I understand, even if that means needing to boot a different, sufficiently-working kernel that does not match the debug information in order to get the /var/crash materials in place and to inspect them. I'm not sure you could do as Konstantin requested based on a non-debug kernel build done the usual way, even with debug information present. Are you using a non-debug kernel? A debug-kernel? You might need to try reproducing with a debug kernel. (But that likely will make builds take longer.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)