From owner-freebsd-threads@freebsd.org Sun Jan 31 11:17:40 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED59FA73334 for ; Sun, 31 Jan 2016 11:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C5A941AB3 for ; Sun, 31 Jan 2016 11:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0VBHe49029576 for ; Sun, 31 Jan 2016 11:17:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Sun, 31 Jan 2016 11:17:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2016 11:17:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #21 from rblayzor@inoc.net --- After a couple of day s like clockwork, I got another crash. I thought this= one looked like a normal Dovecot bug and reported it to the list to which Timo responded:=20 "There's no way it can crash here, unless something's badly broken. Either a buggy compiler or maybe memory corruption earlier in the process. Can you reproduce it? Can you run it via valgrind?" Core was generated by `dovecot'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000800910c74 in io_loop_time_refresh () at ioloop.c:635 635 ioloop.c: No such file or directory. in ioloop.c (gdb) bt full #0 0x0000000800910c74 in io_loop_time_refresh () at ioloop.c:635 No locals. #1 0x0000000800910961 in io_loop_timeouts_start_new (ioloop=3D0x801421080)= at ioloop.c:436 to_idx =3D (struct timeout * const *) 0x801419880 #2 0x0000000800910915 in io_loop_handler_run (ioloop=3D0x801421080) at ioloop.c:606 No locals. #3 0x000000080091073f in io_loop_run (ioloop=3D0x801421080) at ioloop.c:583 No locals. #4 0x000000080086aaab in master_service_run (service=3D0x80141b140, callba= ck=3D0) at master-service.c:640 No locals. #5 0x0000000000406509 in main (argc=3D3, argv=3D0x7fffffffb4c0) at main.c:= 888 set =3D (struct master_settings *) 0x801429110 error =3D 0x800caa2c3 "H\203=C3=84 ]=C3=83\017\037\200" doveconf_arg =3D 0x0 orig_info_callback =3D (failure_callback_t *) 0x8008f0220 orig_debug_callback =3D (failure_callback_t *) 0x8008f0220 foreground =3D false ask_key_pass =3D false i =3D 3 c =3D -1 doubleopts =3D Current language: auto; currently minimal (gdb) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon Feb 1 12:51:32 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BDECA7454B for ; Mon, 1 Feb 2016 12:51:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 256FB616 for ; Mon, 1 Feb 2016 12:51:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u11CpVeF052520 for ; Mon, 1 Feb 2016 12:51:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 01 Feb 2016 12:51:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 12:51:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #22 from rblayzor@inoc.net --- When I see processes core dump along with the line message 'Failed to write core file for process XXX (error 14)", that is generally where I will see an incomplete core file with a "cannot access memory". So that may be related = to the bug you mentioned. I do get occasional good core dumps, as noted above. At this time I cannot = run stable to patch the core dump issue. I did however install clang3.6. I have rebuilt the ports we are seeing prob= lems with against that just to rule it out. That is however, very unlikely.=20 I will continue to gather more dumps if they produce valid cores. If there = is a patch I can rebuild against that fixes the "error 14" issue, I'll give is a whirl. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon Feb 1 13:06:10 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72BFFA74AEE for ; Mon, 1 Feb 2016 13:06:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5BFBABE1 for ; Mon, 1 Feb 2016 13:06:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u11D6Ava015333 for ; Mon, 1 Feb 2016 13:06:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 01 Feb 2016 13:06:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 13:06:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 Konstantin Belousov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kib@FreeBSD.org --- Comment #23 from Konstantin Belousov --- (In reply to rblayzor from comment #22) This smells like a pager error. You mentioned somewhere that the images for the programs which failed are located on NFS volume ? What is in dmesg ? I believe that pager somehow fails to page in, and this causes both access errors in the live processes (SIGSEGV) and EFAULT when kernel reads process mappings to dump them into core files. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon Feb 1 13:34:33 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAEB5A97411 for ; Mon, 1 Feb 2016 13:34:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C8411975 for ; Mon, 1 Feb 2016 13:34:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u11DYXxt072622 for ; Mon, 1 Feb 2016 13:34:33 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 01 Feb 2016 13:34:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 13:34:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #24 from rblayzor@inoc.net --- (In reply to Konstantin Belousov from comment #23) These VM's boot diskless and mount their root file system as RO.=20 Trying to mount root from nfs: []... NFS ROOT: 10.0.0.211:/vol/bsdroot/v0f Where "v0f" is nothing more than subdir for a new rootfs. "bsdroot" is jus= t a NetApp volume RO volume from these hosts. There is a VMDK (virtual disk) that is mounted at boot time; which is a thin provision disk 10GB in size. 2GB a 2Gbps swap partition and a 8Gb scratch = disk for things like /var or /var/tmp da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device bootpc_init: wired to interface 'vmx0' da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 10240MB (20971520 512 byte sectors: 255H 63S/T 1305C) da0: quirks=3D0x40 The port binaries are built on a another FreeBSD 10.2 platform (same VM con= fig) and then we rsync /usr/local to be part of the rootfs. We always keep the port builds and the base system fresh and in sync. Typic= ally when we want roll out a new version of FreeBSD we'll upgrade the build VM first, then create a new subdir for rootfs and install there. Followed by a complete port rebuild and resync. This has always worked for us since FreeB= SD 8.x just fine... Only recently we've seen these random, strange, unexplainable core dumps. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon Feb 1 17:17:10 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EB11A974FF for ; Mon, 1 Feb 2016 17:17:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2040615F9 for ; Mon, 1 Feb 2016 17:17:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u11HH9Ya006196 for ; Mon, 1 Feb 2016 17:17:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 01 Feb 2016 17:17:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 17:17:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #25 from Konstantin Belousov --- (In reply to rblayzor from comment #24) Of course, I mean dmesg after the mysterious seg fault. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon Feb 1 21:05:37 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2082A97697 for ; Mon, 1 Feb 2016 21:05:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2F07112F for ; Mon, 1 Feb 2016 21:05:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u11L5b7s097941 for ; Mon, 1 Feb 2016 21:05:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 01 Feb 2016 21:05:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 21:05:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #26 from rblayzor@inoc.net --- (In reply to Konstantin Belousov from comment #25) I think there are two different issues maybe going on here. My original bug report should of been more specific in that application/processes that apparently ran fine prior to upgrading to 10.2-RELEASE suddenly seem to be = core dumping for some reason. When these processes core dump, sometimes they are leaving good core files, sometimes they are not. (perhaps most of the time). I'm seeing a something = in common with the bug John brought up. Usually when I see something like: Failed to write core file for process exim (error 14) pid 8298 (exim), uid 26: exited on signal 11 The core file is incomplete, and some of the last messages I see in the core file is "cannot access memory", or something long those lines. We'll chalk these up to maybe the bug he mentioned with the system not dumping these co= re files. I am telling the system to drop all cores to a specific directory th= at has plenty of space and is r/w. The main problem however... I have to collect more core dumps on. (if they = can drop core successfully). I posted one previously that just happened the oth= er day that Timo Sirainen said should not be possible due to the output from t= he backtrace. I'm still waiting for more segfaults and still post those backtraces when t= hey are available. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Fri Feb 5 12:34:59 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3351CA778BD for ; Fri, 5 Feb 2016 12:34:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 24A0B14A4 for ; Fri, 5 Feb 2016 12:34:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15CYw3X095194 for ; Fri, 5 Feb 2016 12:34:59 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Fri, 05 Feb 2016 12:34:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 12:34:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #27 from rblayzor@inoc.net --- Just had one of the VM's sig11 on Dovecot last night... This one is giving = me the cannot access memory error ... This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `dovecot'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 memcpy () at bcopy.S:65 65 bcopy.S: No such file or directory. in bcopy.S (gdb) bt full #0 memcpy () at bcopy.S:65 No locals. #1 0x0000000800915484 in sig_handler (signo=3D20, si=3D0x7fffffffa9a0, context=3D0x7fffffffa630) at lib-signals.c:132 h =3D (struct signal_handler *) 0x8014198e0 saved_errno =3D 14 c =3D 0 '\0' tmp_si =3D {si_signo =3D 0, si_errno =3D 0, si_code =3D 0, si_pid = =3D 0, si_uid =3D 0, si_status =3D 0, si_addr =3D 0x0, si_value =3D {sival_int =3D 0, sival_ptr =3D 0x0, sigval= _int =3D 0, sigval_ptr =3D 0x0}, _reason =3D {_fault =3D {_trapno =3D 0}, _timer =3D {_timerid =3D 0, _ove= rrun =3D 0}, _mesgq =3D {_mqd =3D 0}, _poll =3D {_band =3D 0}, __spare__ =3D {__spare1__ =3D 0, __spare2__ =3D 0x7fffffffa5c8}}} #2 No symbol table info available. #3 0x0000000000411fc8 in _fini () No symbol table info available. #4 0x000000080061c9a8 in objlist_call_fini () at /.amd_mnt/juno/pub/FreeBSD/10.2-RELEASE/libexec/rtld-elf/rtld.c:2359 list =3D (Objlist *) Cannot access memory at address 0x0 Current language: auto; currently asm (gdb) x/i $rip 0x800cd1a76 : rep movsq %ds:(%rsi),%es:(%rdi) (gdb) info registers rax 0x800b893f0 34371834864 rbx 0x7fffffffaef8 140737488334584 rcx 0xa 10 rdx 0x50 80 rsi 0x7fffffffa9a0 140737488333216 rdi 0x800b893f0 34371834864 rbp 0x7fffffffa610 0x7fffffffa610 rsp 0x7fffffffa578 0x7fffffffa578 r8 0xffff800800b8ea50 -140703116498352 r9 0xfffff8000a811000 -8795916791808 r10 0x800636200 34366251520 r11 0x246 582 r12 0x800636200 34366251520 r13 0x800834830 34368342064 r14 0x7fffffffaef8 140737488334584 r15 0x0 0 rip 0x800cd1a76 0x800cd1a76 eflags 0x10206 66054 cs 0x43 67 ss 0x3b 59 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (gdb) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Fri Feb 5 13:18:31 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B2EE9D94EE for ; Fri, 5 Feb 2016 13:18:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3CB46F6C for ; Fri, 5 Feb 2016 13:18:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15DIUbn096503 for ; Fri, 5 Feb 2016 13:18:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Fri, 05 Feb 2016 13:18:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 13:18:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #28 from rblayzor@inoc.net --- On that last crash, upon starting Dovecot-- this message appeared. First ti= me we've seen it, but maybe due to all the debug symbols and misc debugs enabl= ed: Starting dovecot. Last died with error (see error log for more information): gettimeofday(): = Bad address --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Fri Feb 5 20:15:41 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E686EA778D1 for ; Fri, 5 Feb 2016 20:15:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D72D0A84 for ; Fri, 5 Feb 2016 20:15:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15KFfTZ097358 for ; Fri, 5 Feb 2016 20:15:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Fri, 05 Feb 2016 20:15:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 20:15:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #29 from John Baldwin --- That would imply gettimeofday() is failing with EFAULT. That seems like a really odd error. That should only be failing if the address passed to gettimeofday() is bad, but that seems weird. Maybe try setting kern.timecounter.fast_gettime=3D0 as a test? Some of your faults have addresses in the shared page that holds the fast timecounter st= uff. It's a bit of a hail mary, but if it works then it narrows down what we ne= ed to look at to find the real bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Fri Feb 5 20:26:11 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10157A77C18 for ; Fri, 5 Feb 2016 20:26:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 010CA11A7 for ; Fri, 5 Feb 2016 20:26:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15KQAev018791 for ; Fri, 5 Feb 2016 20:26:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Fri, 05 Feb 2016 20:26:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 20:26:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #30 from Konstantin Belousov --- (In reply to John Baldwin from comment #29) Fast gettimeofday cannot fail with EFAULT, there is no such return path. An access to the wrong memory from fast gettimeofday code would result in real fault. --=20 You are receiving this mail because: You are the assignee for the bug.=