Skip site navigation (1)Skip section navigation (2) 
Date:      Mon, 16 May 2016 21:20:28 +0000 (UTC)
From:      Greg Quinlan <gwq_uk@yahoo.com>
To:        Freebsd current <freebsd-current@freebsd.org>
Subject:   Re: Kernel panic - help
Message-ID:  <1701232627.3438539.1463433628224.JavaMail.yahoo@mail.yahoo.com>
In-Reply-To: <474864611.2829850.1463347476264.JavaMail.yahoo@mail.yahoo.com>
References:  <474864611.2829850.1463347476264.JavaMail.yahoo.ref@mail.yahoo.com> <474864611.2829850.1463347476264.JavaMail.yahoo@mail.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help 
And again, at almost the same time ... kernel panic!!!
May 17 03:06:18 dns0 syslogd: kernel boot file is /boot/kernel/kernel
May 17 03:06:18 dns0 kernel: panic: softdep_deallocate_dependencies: dangli=
ng deps
May 17 03:06:18 dns0 kernel: cpuid =3D 1
May 17 03:06:18 dns0 kernel: KDB: stack backtrace:
May 17 03:06:18 dns0 kernel: #0 0xffffffff80ae4737 at kdb_backtrace+0x67
May 17 03:06:18 dns0 kernel: #1 0xffffffff80a9f502 at vpanic+0x182
May 17 03:06:18 dns0 kernel: #2 0xffffffff80a9f373 at panic+0x43
May 17 03:06:18 dns0 kernel: #3 0xffffffff80d5fa81 at softdep_deallocate_de=
pendencies+0x71
May 17 03:06:18 dns0 kernel: #4 0xffffffff80b47eb9 at brelse+0x1b9
May 17 03:06:18 dns0 kernel: #5 0xffffffff80b456c2 at bufwrite+0x182
May 17 03:06:18 dns0 kernel: #6 0xffffffff80d8245c at ffs_write+0x39c
May 17 03:06:18 dns0 kernel: #7 0xffffffff81093459 at VOP_WRITE_APV+0x139
May 17 03:06:18 dns0 kernel: #8 0xffffffff80dce82a at vnode_pager_generic_p=
utpages+0x38a
May 17 03:06:18 dns0 kernel: #9 0xffffffff810953b6 at VOP_PUTPAGES_APV+0xa6
May 17 03:06:18 dns0 kernel: #10 0xffffffff80dcc4be at vnode_pager_putpages=
+0xfe
May 17 03:06:18 dns0 kernel: #11 0xffffffff80dc19ca at vm_pageout_flush+0xe=
a
May 17 03:06:18 dns0 kernel: #12 0xffffffff80db848a at vm_object_page_colle=
ct_flush+0x21a
May 17 03:06:18 dns0 kernel: #13 0xffffffff80db81c7 at vm_object_page_clean=
+0x197
May 17 03:06:18 dns0 kernel: #14 0xffffffff80db7c79 at vm_object_terminate+=
0x99
May 17 03:06:18 dns0 kernel: #15 0xffffffff80dccb96 at vnode_destroy_vobjec=
t+0xc6
May 17 03:06:18 dns0 kernel: #16 0xffffffff80d8a336 at ufs_reclaim+0x26
May 17 03:06:18 dns0 kernel: #17 0xffffffff810944b6 at VOP_RECLAIM_APV+0xa6
May 17 03:06:18 dns0 kernel: Uptime: 16h54m12s
May 17 03:06:18 dns0 kernel: Dumping 896 out of 8113 MB:..2%..11%..22%..31%=
..42%..52%..61%..72%..81%..92%

    On Monday, 16 May 2016, 7:24, Greg Quinlan <gwq_uk@yahoo.com> wrote:
=20

 FreeBSD dns0 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r299817M: Sun May 15 18:=
55:50 AEST 2016=C2=A0 =C2=A0 root@dns0:/usr/obj/usr/src/sys/GENERIC=C2=A0 a=
md64

=3D=3D=3D8<=3D=3D=3D
May 16 03:03:48 dns0 syslogd: kernel boot file is /boot/kernel/kernel
May 16 03:03:48 dns0 kernel: panic: softdep_deallocate_dependencies: dangli=
ng deps
May 16 03:03:48 dns0 kernel: cpuid =3D 0
May 16 03:03:48 dns0 kernel: KDB: stack backtrace:
May 16 03:03:48 dns0 kernel: #0 0xffffffff80ae4737 at kdb_backtrace+0x67
May 16 03:03:48 dns0 kernel: #1 0xffffffff80a9f502 at vpanic+0x182
May 16 03:03:48 dns0 kernel: #2 0xffffffff80a9f373 at panic+0x43
May 16 03:03:48 dns0 kernel: #3 0xffffffff80d5fa81 at softdep_deallocate_de=
pendencies+0x71
May 16 03:03:48 dns0 kernel: #4 0xffffffff80b47eb9 at brelse+0x1b9
May 16 03:03:48 dns0 kernel: #5 0xffffffff80b456c2 at bufwrite+0x182
May 16 03:03:48 dns0 kernel: #6 0xffffffff80d8245c at ffs_write+0x39c
May 16 03:03:48 dns0 kernel: #7 0xffffffff81093459 at VOP_WRITE_APV+0x139
May 16 03:03:48 dns0 kernel: #8 0xffffffff80dce82a at vnode_pager_generic_p=
utpages+0x38a
May 16 03:03:48 dns0 kernel: #9 0xffffffff810953b6 at VOP_PUTPAGES_APV+0xa6
May 16 03:03:48 dns0 kernel: #10 0xffffffff80dcc4be at vnode_pager_putpages=
+0xfe
May 16 03:03:48 dns0 kernel: #11 0xffffffff80dc19ca at vm_pageout_flush+0xe=
a
May 16 03:03:48 dns0 kernel: #12 0xffffffff80db848a at vm_object_page_colle=
ct_flush+0x21a
May 16 03:03:48 dns0 kernel: #13 0xffffffff80db81c7 at vm_object_page_clean=
+0x197
May 16 03:03:48 dns0 kernel: #14 0xffffffff80db7c79 at vm_object_terminate+=
0x99
May 16 03:03:48 dns0 kernel: #15 0xffffffff80dccb96 at vnode_destroy_vobjec=
t+0xc6
May 16 03:03:48 dns0 kernel: #16 0xffffffff80d8a336 at ufs_reclaim+0x26
May 16 03:03:48 dns0 kernel: #17 0xffffffff810944b6 at VOP_RECLAIM_APV+0xa6
May 16 03:03:48 dns0 kernel: Uptime: 7h31m3s
May 16 03:03:48 dns0 kernel: Dumping 953 out of 8113 MB:..2%..11%..21%..31%=
..41%..51%..61%..71%..81%..91%=C2=A0

Not sure if the above is useful, seems to happen early morning.

Thanks

Greg

  
From owner-freebsd-current@freebsd.org  Tue May 17 06:24:02 2016
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id B89B6B3EA32
 for <freebsd-current@mailman.ysv.freebsd.org>;
 Tue, 17 May 2016 06:24:02 +0000 (UTC) (envelope-from avg@FreeBSD.org)
Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140])
 by mx1.freebsd.org (Postfix) with ESMTP id D623B1F34;
 Tue, 17 May 2016 06:24:01 +0000 (UTC) (envelope-from avg@FreeBSD.org)
Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua
 [212.40.38.100])
 by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA12078;
 Tue, 17 May 2016 09:23:54 +0300 (EEST)
 (envelope-from avg@FreeBSD.org)
Received: from localhost ([127.0.0.1])
 by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD))
 id 1b2YQ5-000Ijh-Vg; Tue, 17 May 2016 09:23:54 +0300
To: FreeBSD Current <freebsd-current@FreeBSD.org>,
 Edward Tomasz Napierala <trasz@FreeBSD.org>
From: Andriy Gapon <avg@FreeBSD.org>
Subject: panic: destroying non-empty racct: 2113536 allocated for resource 4
Message-ID: <d76b4949-c2a2-d003-af73-df07ea6bf248@FreeBSD.org>
Date: Tue, 17 May 2016 09:22:57 +0300
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101
 Thunderbird/45.1.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>;
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 17 May 2016 06:24:02 -0000


To be fair I got this panic after some exotic sequence of events: running
poudriere, sending SIGSTOP to one of build processes, forgetting about it,
seeing poudriere timeout that job, sending SIGCONT...

This is amd64 head r297350.

Some details:
(kgdb) bt
#0  doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:295
#1  0xffffffff8062d7ef in kern_reboot (howto=<optimized out>) at
/usr/src/sys/kern/kern_shutdown.c:363
#2  0xffffffff8062de38 in vpanic (fmt=<optimized out>, ap=0xfffffe0519b73920) at
/usr/src/sys/kern/kern_shutdown.c:639
#3  0xffffffff8062db43 in panic (fmt=<unavailable>) at
/usr/src/sys/kern/kern_shutdown.c:572
#4  0xffffffff8061ef1c in racct_destroy_locked (racctp=<optimized out>) at
/usr/src/sys/kern/kern_racct.c:478
#5  0xffffffff8061ee45 in racct_destroy (racct=0xfffff802f6301518) at
/usr/src/sys/kern/kern_racct.c:495
#6  0xffffffff805fdd3c in prison_racct_free_locked (prr=0xfffff802f6301400) at
/usr/src/sys/kern/kern_jail.c:4564
#7  0xffffffff805fdc8d in prison_racct_free (prr=0xfffff802f6301400) at
/usr/src/sys/kern/kern_jail.c:4583
#8  0xffffffff805fddee in prison_racct_detach (pr=0xfffff802b0730000) at
/usr/src/sys/kern/kern_jail.c:4658
#9  0xffffffff805fb2cb in prison_deref (pr=<optimized out>, flags=3) at
/usr/src/sys/kern/kern_jail.c:2663
#10 0xffffffff805fca25 in prison_remove_one (pr=<optimized out>) at
/usr/src/sys/kern/kern_jail.c:2358
#11 0xffffffff805fc8e4 in sys_jail_remove (td=<optimized out>, uap=<optimized
out>) at /usr/src/sys/kern/kern_jail.c:2313
#12 0xffffffff80820ddd in syscallenter (td=0xfffff801146019e0,
sa=0xfffffe0519b73b80) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135
#13 0xffffffff808209af in amd64_syscall (td=0xfffff801146019e0, traced=0) at
/usr/src/sys/amd64/amd64/trap.c:943

RACCT_RSS is 4.

(kgdb) p *prr
$5 = {
  prr_next = {
    le_next = 0xfffff80382fe4400,
    le_prev = 0xfffff8017ac90600
  },
  prr_name = "basejail-default-job-03", '\000' <repeats 232 times>,
  prr_refcount = 0,
  prr_racct = 0xfffff802e3f520b0
}
(kgdb) p *prr->prr_racct
$6 = {
  r_resources = {13884177072, 0, 0, 0, 2113536, 0 <repeats 14 times>,
13611325009, 0},
  r_rule_links = {
    lh_first = 0x0
  }
}

Could it be that somehow the CONT'd process failed to deduct its resources from
the jail's resources because the jail was already marked for destruction or
something like that?

-- 
Andriy Gapon

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1701232627.3438539.1463433628224.JavaMail.yahoo>