Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 24 Nov 2017 00:54:41 -0800
From:      Mark Millard <markmi@dsl-only.net>
To:        FreeBSD Current <freebsd-current@freebsd.org>, freebsd-hackers <freebsd-hackers@freebsd.org>
Subject:   head -r325997 panics: dump/restore: restore got ufs_dirbad ino . . .: mangled entry; cp -ax got: ffs_valloc: dup alloc (after some cylinder checksum failures)
Message-ID:  <F090C1F8-00DA-4436-8B77-D0784CB8CDCF@dsl-only.net>

next in thread | raw e-mail | index | archive | help
head -r323997 context, non-debug kernel (with
debug symbols).

This turned out to be for a USB SSD that was
dying (Retired Block Count had grown rather
large, other things looking okay).

I do not know if panics are an intended
result for the messed up data or not.


The panic information follows.

The USB ssd with /dev/gpt/FBSDUSSDroot on it had
just been freshly constructed via bsdconfig, UFS
with softupdate enabled but journaling disabled.

# mount -o noatime /dev/gpt/FBSDUSSDroot /mnt
# dump -C16 -b64 -0aL -f - / | (cd /mnt && restore -rf -)
  DUMP: Date of this level 0 dump: Thu Nov 23 16:26:00 2017
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping snapshot of /dev/gpt/FBSDFSSDroot (/) to standard output
  DUMP: mapping (Pass I) [regular files]
  DUMP: Cache 16 MB, blocksize =3D 65536
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 140989948 tape blocks.
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
warning: ./.snap: File exists
expected next file 9, got 8

Manually transcribed from the console screen
for the crash:

panic: ufs_dirbad ino 308183317 at offset 0: mangled entry.
cpuid =3D 21
time =3D 1511483271
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame =
0xfffffe1c3449a2b0
vpanic() at panic+0x19c/frame 0xfffffe1c3449a330
panic() at panic+0x43/frame 0xfffffe1c3449a390
ufs_lokup_ino at ufs_lookup_ino+0xe7f/frame 0xfffffe1c3449a4a0
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x77c/frame =
0xffffe1c3449a4d0
vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe1c3449a530
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x7c/frame 0xfffffe1c3449a560
lookup() at lookup+0x6c1/frame 0xfffffe1c3449a600
namei() at namei+0x48c/frame 0xfffffe1c3449a6d0
vn_open_cred() at vn_open_cred+0xd8/frame 0xfffffe1c3449a810
kern_openat at kern_openat+0x212/frame 0xfffffe1c3449a980
amd64_syscall() at amd64_syscall+0x9f7/frame 0xfffffe1c3499aab0
Xfast_syscall() at XFast_syscall+0xfb/frame 0xfffffe1c3499aab0
--- syscall (449, FreEBSD ELF64, sys_openat), rip =3D 0x80093be2a, rsp =3D=
 0x7fffffffd518, rpb=3D0x7fffffffd600 ---
KDB: enter: panic
[ thread pid 915 tid 100328 ]
Stopped at kdb_enter_0x3b: movq $0,kbd_why
db>


A retry via (newfs copied from a dumpfs -m output):

# newfs -O 2 -U -a 4 -b 32768 -d 32768 -e 4096 -f 4096 -g 16384 -h 64 -i =
8192 -k 6408 -m 8 -o time -s 905969664 /dev/gpt/FBSDUSSDroot=20
/dev/gpt/FBSDUSSDroot: 442368.0MB (905969664 sectors) block size 32768, =
fragment size 4096
	using 707 cylinder groups of 626.09MB, 20035 blks, 80256 inodes.
	with soft updates
super-block backups (for fsck_ffs -b #) at:
. . .
# mount -o noatime /dev/gpt/FBSDUSSDroot /mnt
FBSDFSSD# dump -C16 -b64 -0aL -f - / | (cd /mnt && restore -rf -)
  DUMP: Date of this level 0 dump: Thu Nov 23 17:02:18 2017
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping snapshot of /dev/gpt/FBSDFSSDroot (/) to standard output
  DUMP: mapping (Pass I) [regular files]
  DUMP: Cache 16 MB, blocksize =3D 65536
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 141064070 tape blocks.
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
warning: ./.snap: File exists
expected next file 5, got 4
  DUMP: 33.57% done, finished in 0:09 at Thu Nov 23 17:17:12 2017


did it again: bad dir ino 11316224 . (I'm not transcribing
this screen but it is very similar to the earlier one.)


Then, trying a cp -ax instead lead to a different
panic (crash info transcribed from screenshot of
console). . .

# newfs -O 2 -U -a 4 -b 32768 -d 32768 -e 4096 -f 4096 -g 16384 -h 64 -i =
8192 -k 6408 -m 8 -o time -s 905969664 /dev/gpt/FBSDUSSDroot
/dev/gpt/FBSDUSSDroot: 442368.0MB (905969664 sectors) block size 32768, =
fragment size 4096
	using 707 cylinder groups of 626.09MB, 20035 blks, 80256 inodes.
	with soft updates
super-block backups (for fsck_ffs -b #) at:
. . .
# mount -o noatime /dev/gpt/FBSDUSSDroot /mnt
# cp -ax / /mnt

Eventually on the console got:

. . .
UFS /dev/gpt/FBSDUSSDroot (/mnt) cylinder checksum failed: cg 475, =
cgp:0x782bac7c !=3D bp: 0x3c6d091f
UFS /dev/gpt/FBSDUSSDroot (/mnt) cylinder checksum failed: cg 477, =
cgp:0x782bac7c !=3D bp: 0xcbfd1084
mode =3D 0116600, inum =3D 38603264, fs =3D /mnt
panic: ffs_valloc: dup alloc
cpuid =3D 27
time =3D 1511490715
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame =
0xfffffe1c344fe400
vpanic() at vpanic+0x19c/frame 0xfffffe1c344fe480
panic() at panic+0x43/frame 0xfffffe1c344fe4e0
ffs_valloc() at ffs_valloc+0x85c/frame 0xffffe1c344fe570
ufs_mkdir() at ufs_mkdir+0xcc/frame 0xfffffe1c344fe730
VOP_MKDIR_APV() at VOP_MKDIR_APV+0x82/frame 0xfffffe1c344fe760
kern_mkdirat() at kern_mkdirat+0x1e2/frame 0xfffffe1c344fe980
amd64_syscall() at amd64_syscall+0x9f7/frame 0xfffffe1c344feab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe1c344feab0
--- syscall (136, FreBSD ELF64, sys_mkdir), rip =3D 0x80099a78a, rsp =3D =
0x7fffffffd838, rbp =3D 0x7fffffffdb30 ---
KDB: enter: panic
[ thread pid 966 tid 100348 ]
Stopped at kdb_enter+0x3b: movq $0,kdb_why
db>




=3D=3D=3D
Mark Millard
markmi at dsl-only.net




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F090C1F8-00DA-4436-8B77-D0784CB8CDCF>