From owner-freebsd-stable@FreeBSD.ORG Wed Oct 15 17:23:33 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1815AA62 for ; Wed, 15 Oct 2014 17:23:33 +0000 (UTC) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D87635EF for ; Wed, 15 Oct 2014 17:23:32 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id r10so1813885igi.14 for ; Wed, 15 Oct 2014 10:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=n6WDiD2s+8rUhAvdc72PFP0tX9yo3Z+ljeeYqlyhw04=; b=bQyKLoWXX/F1FXE3uqAsi4y3V/NsJ5ycMXkS9goDp1r9NQ3e2ydIoXLNjFkwOX9vE9 duwueU1uamUWuYS7RR4vErcLPZQfOyDEapc8EU5IhbZfaYmzSxPBXLPXUlwchk0teSWv PqD47J1dk/GRllXO6hKmFyk9bix4FSv1iB8xn3p22uzPjtpnflFkQFmRCCHGWEa1TeK2 J9Y15ws31hPApizyBD+asV7QA47hYXsfr0Zv1MBfg3dSKre2VaALva7YmmguSnjlZ3B/ kloE4oAvJ5Oq1MFCgIgBSeEFXhHdyUZa5heL5oysDBk6RbEvpQKil9WpUdhYIRgictcl SGyA== MIME-Version: 1.0 X-Received: by 10.42.26.14 with SMTP id d14mr12621945icc.5.1413393812184; Wed, 15 Oct 2014 10:23:32 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.107.11.141 with HTTP; Wed, 15 Oct 2014 10:23:32 -0700 (PDT) In-Reply-To: <7479DC25-4451-4940-AFE7-7C81D08206D4@csperkins.org> References: <7479DC25-4451-4940-AFE7-7C81D08206D4@csperkins.org> Date: Wed, 15 Oct 2014 10:23:32 -0700 X-Google-Sender-Auth: lAXbJzYikgAejE8T0uIWa0eILRE Message-ID: Subject: Re: System hang on shutdown when running freebsd-update From: Kevin Oberman To: Colin Perkins Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD-STABLE Mailing List X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Oct 2014 17:23:33 -0000 On Wed, Oct 15, 2014 at 2:40 AM, Colin Perkins wrote: > On 14 Oct 2014, at 18:09, Kevin Oberman wrote: > > I thought that this was just a fluke, but it has now happened three > times, > > so I guess it's now out of the "fluke" class. > > > > I have upgraded several times recently to each 10.1 BETA and RC. After > the > > first install pass t install the kernel and modules, the system shutdow= n > > freezes at the very end. I see the buffers synced to the disks and get > the > > "All buffers synced" message. Then it just hangs. The disks are not > marked > > as clean and are fscked after a reset and boot. > > > > There is not much between the "All buffers synced" message and the call > to > > vfs_unmountall(), so I suspect it is hanging in that call. I admit that= I > > am pretty much lost whenever I look at the VFS code and I have not put = a > > lot of effort going further. Just hoping that someone familiar with it > > might have an idea. > > > > I have tried several reboots and all run normally. The problem only see= ms > > to appear when upgrading the OS. It happened repeatedly when I tried to > > reboot before doing the second "install" pass of freebsd-update, but no= t > > after, so the kernel and world are not in sync. I am baffled as to what > > could be going on, but it means I need to be at the system (a baby > server) > > when I upgrade, but not every time I upgrade. I know it happened on the > > 10.0-RELEAASE to 10.1-BETA1 and 10.1-RC1 to 10.1-RC2 upgrades. > > > > Has anyone else seen this? > > I=E2=80=99m seeing the same behaviour, most recently when moving to 10.1-= RC1 > (haven=E2=80=99t gone to -RC2 yet). The system is: > > FreeBSD 10.1-RC1 #0 r272463: Fri Oct 3 01:47:10 UTC 2014 > root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > CPU: AMD Opteron(TM) Processor 6274 (2200.05-MHz K8-clas= s > CPU) > Origin =3D "AuthenticAMD" Id =3D 0x600f12 Family =3D 0x15 Model =3D 0x= 1 > Stepping =3D 2 > > Features=3D0x178bfbff > > Features2=3D0x1e98220b > AMD Features=3D0x2e500800 > AMD > Features2=3D0x1c9bfff > TSC: P-state invariant, performance statistics > real memory =3D 549755813888 (524288 MB) > avail memory =3D 534559084544 (509795 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: <041112 APIC1739> > FreeBSD/SMP: Multiprocessor System Detected: 64 CPUs > FreeBSD/SMP: 4 package(s) x 16 core(s) > =E2=80=A6 > > I do have IPMI loaded, unlike the other reports. > > -- > Colin Perkins > https://csperkins.org/ > Paul Koch replied privately with a pointer to a seemingly unrelated message he sent to stable last month. Take a look at the several paragraphs at the end starting with "On a side note". I'm suspicious that the generation of the large upgrade on /var during the "upgrade" pass is causing the delay. It fits pretty well and, in normal operation, my server would never see this issue at all. https://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D326083+0+/usr/local/www/db/= text/2014/freebsd-stable/20140907.freebsd-stable Aside from fsyncing the files, I suspect just running "upgrade" waiting for a long time before doing reboot might prevent it from happening. I is likely relevant that the single partition on hte system is a 500GB SU+J UFS= . I need to research a bit on how freebsd does things as well as possible interaction with the large SU+J partition. I was already uncomfortable about the SU+J but went with it due to the time it would otherwise take to fsck the 500GB disk. -- R. Kevin Oberman, Network Engineer, Retired rkoberman@gmail.com