From owner-freebsd-net@freebsd.org Sun Jun 19 11:35:45 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D2AFA7A090 for ; Sun, 19 Jun 2016 11:35:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D3AE2786 for ; Sun, 19 Jun 2016 11:35:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JBZirj065366 for ; Sun, 19 Jun 2016 11:35:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Sun, 19 Jun 2016 11:35:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 11:35:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jun 19 11:39:25 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7ED49A7A2DF for ; Sun, 19 Jun 2016 11:39:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6EFA72A5C for ; Sun, 19 Jun 2016 11:39:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JBdPoM071020 for ; Sun, 19 Jun 2016 11:39:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210340] ipv6 alias, don't respond after a time Date: Sun, 19 Jun 2016 11:39:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 11:39:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210340 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jun 19 18:01:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 543D3A7A2AB for ; Sun, 19 Jun 2016 18:01:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43FA02665 for ; Sun, 19 Jun 2016 18:01:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JI1t1o062206 for ; Sun, 19 Jun 2016 18:01:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Sun, 19 Jun 2016 18:01:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: allanjude@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 18:01:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #1 from Allan Jude --- may be related to the change discussed in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D208067 which was commit= ted as r297403 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jun 19 20:08:35 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6313FA795FF for ; Sun, 19 Jun 2016 20:08:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5310C19A8 for ; Sun, 19 Jun 2016 20:08:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JK8ZMT037912 for ; Sun, 19 Jun 2016 20:08:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Sun, 19 Jun 2016 20:08:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 20:08:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 Mark Johnston changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |markj@FreeBSD.org --- Comment #2 from Mark Johnston --- (In reply to Allan Jude from comment #1) It's related in that it touched the line of code at which we're crashing, b= ut without the change that bcopy was bogus: lle->ll_addr is a char *, so bcopy(&lle->ll_addr, LLADDR(sdl), ifp->if_addrlen) just copies the address of the MAC address into sdl, rather than the MAC address itself. The type of lle->ll_addr changed in r292978. Based on the panic, lle->ll_addr is NULL in your case. Did this start occurring after an update? What was the from-revision? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jun 19 20:25:53 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D302A79C1A for ; Sun, 19 Jun 2016 20:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D3DE225F for ; Sun, 19 Jun 2016 20:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JKPqhd076043 for ; Sun, 19 Jun 2016 20:25:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Sun, 19 Jun 2016 20:25:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: allanjude@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 20:25:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #3 from Allan Jude --- sadly, the 'from' version is: r283896: Mon Jun 1 22:08:43 UTC 2015 So literally represents a year worth of changes (the to kernel was Jun 1, 2= 016) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jun 19 20:53:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F30B5A77303 for ; Sun, 19 Jun 2016 20:53:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E33F42E75 for ; Sun, 19 Jun 2016 20:53:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5JKrIqh033609 for ; Sun, 19 Jun 2016 20:53:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Sun, 19 Jun 2016 20:53:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 20:53:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #4 from Mark Johnston --- Could you paste the output of ndp -a? If you're able to create a crash dump, it would be nice to see the output of "p *lle" from in6_lltable_dump_entry()'s frame. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 20 06:53:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35970A7B81E for ; Mon, 20 Jun 2016 06:53:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2538F2090 for ; Mon, 20 Jun 2016 06:53:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5K6rZLD024928 for ; Mon, 20 Jun 2016 06:53:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Mon, 20 Jun 2016 06:53:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 06:53:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ae@FreeBSD.org --- Comment #5 from Andrey V. Elsukov --- Recently I have the same panic when I did `ndp -c`. This is not fresh CURRENT: commit 3a7d342befa3ff4d0e3ecd5baf88e128a41b636f Author: pfg Date: Tue Apr 12 17:23:03 2016 +0000 Replace 0 with NULL for pointers in misc. device drivers. Found with devel/coccinelle. --- Fatal trap 12: page fault while in kernel mode cpuid =3D 2; apic id =3D 02 fault virtual address =3D 0x0 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80ae80d4 stack pointer =3D 0x28:0xfffffe0233953440 frame pointer =3D 0x28:0xfffffe0233953450 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 93382 (ndp) (kgdb) bt #0 doadump (textdump=3D865414752) at pcpu.h:221 #1 0xffffffff803473b6 in db_fncall (dummy1=3D, dummy2=3D, dummy3=3D,=20 dummy4=3D) at /usr/src/sys/ddb/db_command.c:568 #2 0xffffffff80346e59 in db_command (cmd_table=3D) at /usr/src/sys/ddb/db_command.c:440 #3 0xffffffff80346bb4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #4 0xffffffff8034968b in db_trap (type=3D, code=3D) at /usr/src/sys/ddb/db_main.c:251 #5 0xffffffff8078e453 in kdb_trap (type=3D, code=3D, tf=3D) at /usr/src/sys/kern/subr_kdb.c:654 #6 0xffffffff80aea591 in trap_fatal (frame=3D0xfffffe0233953390, eva=3D0) = at /usr/src/sys/amd64/amd64/trap.c:836 #7 0xffffffff80aea7c3 in trap_pfault (frame=3D0xfffffe0233953390, usermode= =3D0) at /usr/src/sys/amd64/amd64/trap.c:691 #8 0xffffffff80ae9d6c in trap (frame=3D0xfffffe0233953390) at /usr/src/sys/amd64/amd64/trap.c:442 #9 0xffffffff80acd411 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #10 0xffffffff80ae80d4 in bcopy () at /usr/src/sys/amd64/amd64/support.S:122 #11 0xffffffff809666fe in in6_lltable_dump_entry (llt=3D, lle=3D0xfffff80173bb2200, wr=3D0xfffffe0233953858) at /usr/src/sys/netinet6/in6.c:2370 #12 0xffffffff80848103 in htable_foreach_lle (llt=3D, f=3D, farg=3D) at /usr/src/sys/net/if_llatbl.c:143 #13 0xffffffff80846bad in lltable_sysctl_dumparp (af=3D, wr=3D) at /usr/src/sys/net/if_llatbl.c:658 #14 0xffffffff808580cb in sysctl_rtsock (oidp=3D, arg1=3D, arg2=3D, req=3D0xfffffe0= 233953858) at /usr/src/sys/net/rtsock.c:1864 #15 0xffffffff80756301 in sysctl_root_handler_locked (oid=3D0xffffffff81170= 638, arg1=3D0xfffffe0233953928, arg2=3D4, req=3D0xfffffe0233953858,=20 tracker=3D0xfffffe02339537d0) at /usr/src/sys/kern/kern_sysctl.c:165 #16 0xffffffff80755ad6 in sysctl_root (arg1=3D, arg2= =3D) at /usr/src/sys/kern/kern_sysctl.c:1841 #17 0xffffffff80756076 in userland_sysctl (td=3D, name=3D0xfffffe0233953920, namelen=3D6, old=3D,=20 oldlenp=3D, inkernel=3D, new= =3D, newlen=3D,=20 retval=3D0xfffffe0233953520, flags=3D0) at /usr/src/sys/kern/kern_sysct= l.c:1944 #18 0xffffffff80755e84 in sys___sysctl (td=3D0xfffff801c81539a0, uap=3D0xfffffe0233953a40) at /usr/src/sys/kern/kern_sysctl.c:1871 #19 0xffffffff80aeaf68 in amd64_syscall (td=3D, traced= =3D0) at subr_syscall.c:135 (kgdb) f 11 #11 0xffffffff809666fe in in6_lltable_dump_entry (llt=3D, lle=3D0xfffff80173bb2200, wr=3D0xfffffe0233953858) at /usr/src/sys/netinet6/in6.c:2370 2370 bcopy(lle->ll_addr, LLADDR(sdl), ifp->if_addrlen); (kgdb) p *lle $1 =3D {lle_next =3D {le_next =3D 0x0, le_prev =3D 0xfffff800039bab08}, r_l= 3addr =3D {addr4 =3D {s_addr =3D 2917007613}, addr6 =3D {__u6_addr =3D { __u6_addr8 =3D 0xfffff80173bb2210 "=EF=BF=BD", __u6_addr16 =3D 0xff= fff80173bb2210, __u6_addr32 =3D 0xfffff80173bb2210}}},=20 r_linkdata =3D 0xfffff80173bb2220 "", r_hdrlen =3D 0 '\0', spare0 =3D 0xfffff80173bb2239 "", r_flags =3D 0, r_skip_req =3D 0, lle_tbl =3D 0xfffff800039bac00,=20 lle_head =3D 0xfffff800039bab08, lle_free =3D 0xffffffff80966920 , la_hold =3D 0xfffff801d1c0ed00, la_numheld =3D 0= ,=20 la_expire =3D 793804, la_flags =3D 64, la_asked =3D 2, la_preempt =3D 0, = ln_state =3D 0, ln_router =3D 0, ln_ntick =3D 0, lle_remtime =3D 0, lle_hittime =3D 0,=20 lle_refcnt =3D 2, ll_addr =3D 0x0, lle_chain =3D {le_next =3D 0x0, le_pre= v =3D 0x0}, lle_timer =3D {c_links =3D {le =3D {le_next =3D 0x0,=20 le_prev =3D 0xfffffe0000c9d030}, sle =3D {sle_next =3D 0x0}, tqe = =3D {tqe_next =3D 0x0, tqe_prev =3D 0xfffffe0000c9d030}}, c_time =3D 3409362326052764,=20 c_precision =3D 268435450, c_arg =3D 0xfffff80173bb2200, c_func =3D 0xffffffff80982620 , c_lock =3D 0x0, c_flags =3D 2, c_ifl= ags =3D 20,=20 c_cpu =3D 0}, lle_lock =3D {lock_object =3D {lo_name =3D 0xffffffff80e9= b1a0 "lle", lo_flags =3D 90374144, lo_data =3D 0, lo_witness =3D 0x0}, rw_lock =3D 1},= =20 req_mtx =3D {lock_object =3D {lo_name =3D 0xffffffff80e9b1a4 "lle req", l= o_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock =3D 4}} (kgdb) p lle->ll_addr $2 =3D 0x0 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 20 06:56:01 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6EA5A7B92A for ; Mon, 20 Jun 2016 06:56:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D723221A3 for ; Mon, 20 Jun 2016 06:56:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5K6u1va028231 for ; Mon, 20 Jun 2016 06:56:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Mon, 20 Jun 2016 06:56:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 06:56:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #6 from Andrey V. Elsukov --- (In reply to Andrey V. Elsukov from comment #5) > Recently I have the same panic when I did `ndp -c`. Probably this was not exact `ndp -c`, but just `ndp -a`. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 20 07:39:24 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BA30A7A93E for ; Mon, 20 Jun 2016 07:39:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 07486174F for ; Mon, 20 Jun 2016 07:39:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0056AA7A93C; Mon, 20 Jun 2016 07:39:24 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3A80A7A93A; Mon, 20 Jun 2016 07:39:23 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E0BC0174C; Mon, 20 Jun 2016 07:39:23 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id u5K7dHNO093644 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 20 Jun 2016 00:39:17 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id u5K7dHwj093643; Mon, 20 Jun 2016 00:39:17 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 20 Jun 2016 00:39:17 -0700 From: Gleb Smirnoff To: rrs@FreeBSD.org Cc: Julien Charbon , hselasky@FreeBSD.org, rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Subject: Re: panic with tcp timers Message-ID: <20160620073917.GI1076@FreeBSD.org> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 07:39:24 -0000 Hi! On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: J> > Comparing stable/10 and head, I see two changes that could J> > affect that: J> > J> > - callout_async_drain J> > - switch to READ lock for inp info in tcp timers J> > J> > That's why you are in To, Julien and Hans :) J> > J> > We continue investigating, and I will keep you updated. J> > However, any help is welcome. I can share cores. Now, spending some time with cores and adding a bunch of extra CTRs, I have a sequence of events that lead to the panic. In short, the bug is in the callout system. It seems to be not relevant to the callout_async_drain, at least for now. The transition to READ lock unmasked the problem, that's why NetflixBSD 10 doesn't panic. The panic requires heavy contention on the TCP info lock. [CPU 1] the callout fires, tcp_timer_keep entered [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); [CPU 2] schedules the callout [CPU 2] tcp_discardcb called [CPU 2] callout successfully canceled [CPU 2] tcpcb freed [CPU 1] unblocks... panic When the lock was WLOCK, all contenders were resumed in a sequence they came to the lock. Now, that they are readers, once the lock is released, readers are resumed in a "random" order, and this allows tcp_discardcb to go before the old running callout, and this unmasks the panic. Exact quote from ktrdump: 0xfffff82089867418 <-- faulty tcpcb 0xfffff82089867728 <-- its tt_keep glebius@piston:~/cores:|>grep 0xfffff82089867418 ktr 19012192 11 scheduled 0xfffff820898677a8 func 0xffffffff80628740 arg 0xfffff82089867418 in 20553.1e3ff819 19012190 11 rescheduled 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 in 20613.04a6193d 19009551 11 rescheduled 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 in 20613.042306cf 19009549 11 scheduled 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 in 20563.0423409f 19009545 11 tcp_newtcpcb: 0xfffff82089867418 18962842 11 tcp_discardcb: free 0xfffff82089867418 18962830 11 tcp_discardcb: 0xfffff82089867418 draincnt 0 18962826 11 failed to stop 0xfffff820898677a8 func 0xffffffff80628740 arg 0xfffff82089867418 18962822 11 cancelled3 0xfffff82089867768 func 0xffffffff806288e0 arg 0xfffff82089867418 18962808 11 cancelled3 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 18962804 11 failed to stop 0xfffff820898676e8 func 0xffffffff80628fa0 arg 0xfffff82089867418 18962792 11 failed to stop 0xfffff820898676a8 func 0xffffffff806291e0 arg 0xfffff82089867418 18962755 11 tcp_discardcb: 0xfffff82089867418 18962742 11 scheduled 0xfffff82089867768 func 0xffffffff806288e0 arg 0xfffff82089867418 in 20632.ffc8d308 18962703 11 cancelled3 0xfffff820898676a8 func 0xffffffff806291e0 arg 0xfffff82089867418 18962695 11 scheduled 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 in 20612.ffc8ea28 18923275 6 tcp_timer_keep: 0xfffff82089867418 18923274 6 callout 0xfffff82089867728 func 0xffffffff80628bb0 arg 0xfffff82089867418 17850361 9 scheduled 0xfffff820898676a8 func 0xffffffff806291e0 arg 0xfffff82089867418 in 20553.5aec0004 -- Totus tuus, Glebius. From owner-freebsd-net@freebsd.org Mon Jun 20 09:56:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 357C5A79808 for ; Mon, 20 Jun 2016 09:56:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 12BFE2106 for ; Mon, 20 Jun 2016 09:56:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0EBB7A79805; Mon, 20 Jun 2016 09:56:04 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E116A79801; Mon, 20 Jun 2016 09:56:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f46.google.com (mail-wm0-f46.google.com [74.125.82.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB2CB2103; Mon, 20 Jun 2016 09:56:03 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f46.google.com with SMTP id v199so61892368wmv.0; Mon, 20 Jun 2016 02:56:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=3R5oeRHX8RbACHoEqf+3o/gYI4UZCkA1gYS8cFHCR0U=; b=LXRMPc35nnTc0GuPTcUFQMNJ2krtP9JxPg10Cjiwamq3K2zJw+1MkAljA1by0CCe4q 3BLTqVgWgzWnnymvzSwquz5M4IU+ndEz6VIj5Uf8618tBH6HNaVYGydJmuv5rTA00JNX /NSmO3K+l7PNNuj1v83hmOL+9svhbP+m+nXZl0a+WwJTpSiheOTfIXHP+1pyDSCw5kvh w30MuD6KuQzHySvdNcb3Thx9xYNESCKyEc+C3pbFMKnTpqwZEkkwa/FV2joFTW/NgchC mfFJO4Av6X6Yme4AWVsfVzSoQBfqpUVMOhGxp4woj6MnbKRDzuNlssKCO6IVqiKSwPyT yCwg== X-Gm-Message-State: ALyK8tLIprB7amIg+uwvZQvTA651fVSjcnlnHTURSPg+nn+7UWCABlaG2gwm5EPuou6RvQ== X-Received: by 10.194.81.72 with SMTP id y8mr14670954wjx.83.1466416561397; Mon, 20 Jun 2016 02:56:01 -0700 (PDT) Received: from [10.100.64.16] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id e5sm62749343wjj.10.2016.06.20.02.56.00 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 20 Jun 2016 02:56:00 -0700 (PDT) Subject: Re: panic with tcp timers To: Gleb Smirnoff , rrs@FreeBSD.org References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> Cc: hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Julien Charbon Message-ID: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> Date: Mon, 20 Jun 2016 11:55:55 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160620073917.GI1076@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="WRcuaKjPkOnTUAdRFNif3MWmTfBRkeXRe" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 09:56:04 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WRcuaKjPkOnTUAdRFNif3MWmTfBRkeXRe Content-Type: multipart/mixed; boundary="oQsT0aQmt8QcIiFDCxI7qFMBav9RqDwaE" From: Julien Charbon To: Gleb Smirnoff , rrs@FreeBSD.org Cc: hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Message-ID: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> Subject: Re: panic with tcp timers References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> In-Reply-To: <20160620073917.GI1076@FreeBSD.org> --oQsT0aQmt8QcIiFDCxI7qFMBav9RqDwaE Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 6/20/16 9:39 AM, Gleb Smirnoff wrote: > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: > J> > Comparing stable/10 and head, I see two changes that could > J> > affect that: > J> >=20 > J> > - callout_async_drain > J> > - switch to READ lock for inp info in tcp timers > J> >=20 > J> > That's why you are in To, Julien and Hans :) > J> >=20 > J> > We continue investigating, and I will keep you updated. > J> > However, any help is welcome. I can share cores. >=20 > Now, spending some time with cores and adding a bunch of > extra CTRs, I have a sequence of events that lead to the > panic. In short, the bug is in the callout system. It seems > to be not relevant to the callout_async_drain, at least for > now. The transition to READ lock unmasked the problem, that's > why NetflixBSD 10 doesn't panic. >=20 > The panic requires heavy contention on the TCP info lock. >=20 > [CPU 1] the callout fires, tcp_timer_keep entered > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); > [CPU 2] schedules the callout > [CPU 2] tcp_discardcb called > [CPU 2] callout successfully canceled > [CPU 2] tcpcb freed > [CPU 1] unblocks... panic >=20 > When the lock was WLOCK, all contenders were resumed in a > sequence they came to the lock. Now, that they are readers, > once the lock is released, readers are resumed in a "random" > order, and this allows tcp_discardcb to go before the old > running callout, and this unmasks the panic. Highly interesting. I should be able to reproduce that (will be useful for testing the corresponding fix). Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1 (success) here) when the callout cancellation is a success _but_ the callout is current running, that should fix it. For the history: It comes back to my old callout question: Does _callout_stop_safe() is allowed to return 1 (success) even if the callout is still currently running; a.k.a. it is not because you successfully cancelled a callout that the callout is not currently runnin= g. We did propose a patch to make _callout_stop_safe() returns 0 (fail) when the callout is currently running: callout_stop() should return 0 when the callout is currently being serviced and indeed unstoppable https://reviews.freebsd.org/differential/changeset/?ref=3D62513&whitespac= e=3Dignore-most But this change impacted too many old code paths and was interesting only for TCP timers and thus was abandoned. My 2 cents. -- Julien --oQsT0aQmt8QcIiFDCxI7qFMBav9RqDwaE-- --WRcuaKjPkOnTUAdRFNif3MWmTfBRkeXRe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXZ72vAAoJEKVlQ5Je6dhxZ3sH/2eFfPP334XgUPWLnMPJ1CeQ gGAz8hshDh9Rmrt7tR+XoG0q8fRanTLP75cOODIYiU51bFYys+0NymLTrsDtjUbF fqRp4cjRznhMEoTiUoCLCIfeIJaer3X5FQDyf1md2Mn+CbtiWswXGr0kH1mnCBwq FBLPwCLF2MEZrXdZImhWCCF+i9KJYXL7gOsu/gCg/5x+JnOK5/Rq4SY6SXvqkBYB p9NKU4E4brZYXatLG4EGaHM4nG16gtw6ZrXmJKfiYMm2en9otRwhbfHfC7xpJG2n ONIMU32WJ095xcOFs+ywUkJ8DFWa0+01AoTy/+OHmIqacrJYMb2hy7mh7O7ylSs= =W+6o -----END PGP SIGNATURE----- --WRcuaKjPkOnTUAdRFNif3MWmTfBRkeXRe-- From owner-freebsd-net@freebsd.org Mon Jun 20 09:58:24 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A23BFA79A02 for ; Mon, 20 Jun 2016 09:58:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8D69923F8 for ; Mon, 20 Jun 2016 09:58:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 85801A799FE; Mon, 20 Jun 2016 09:58:24 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 850C9A799FA; Mon, 20 Jun 2016 09:58:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BB90A23F5; Mon, 20 Jun 2016 09:58:23 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id u5K9wM5B094519 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 20 Jun 2016 02:58:22 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id u5K9wM4P094518; Mon, 20 Jun 2016 02:58:22 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 20 Jun 2016 02:58:22 -0700 From: Gleb Smirnoff To: Julien Charbon Cc: rrs@FreeBSD.org, hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Subject: Re: panic with tcp timers Message-ID: <20160620095822.GJ1076@FreeBSD.org> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 09:58:24 -0000 On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote: J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: J> > J> > Comparing stable/10 and head, I see two changes that could J> > J> > affect that: J> > J> > J> > J> > - callout_async_drain J> > J> > - switch to READ lock for inp info in tcp timers J> > J> > J> > J> > That's why you are in To, Julien and Hans :) J> > J> > J> > J> > We continue investigating, and I will keep you updated. J> > J> > However, any help is welcome. I can share cores. J> > J> > Now, spending some time with cores and adding a bunch of J> > extra CTRs, I have a sequence of events that lead to the J> > panic. In short, the bug is in the callout system. It seems J> > to be not relevant to the callout_async_drain, at least for J> > now. The transition to READ lock unmasked the problem, that's J> > why NetflixBSD 10 doesn't panic. J> > J> > The panic requires heavy contention on the TCP info lock. J> > J> > [CPU 1] the callout fires, tcp_timer_keep entered J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); J> > [CPU 2] schedules the callout J> > [CPU 2] tcp_discardcb called J> > [CPU 2] callout successfully canceled J> > [CPU 2] tcpcb freed J> > [CPU 1] unblocks... panic J> > J> > When the lock was WLOCK, all contenders were resumed in a J> > sequence they came to the lock. Now, that they are readers, J> > once the lock is released, readers are resumed in a "random" J> > order, and this allows tcp_discardcb to go before the old J> > running callout, and this unmasks the panic. J> J> Highly interesting. I should be able to reproduce that (will be useful J> for testing the corresponding fix). J> J> Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1 J> (success) here) when the callout cancellation is a success _but_ the J> callout is current running, that should fix it. J> J> For the history: It comes back to my old callout question: J> J> Does _callout_stop_safe() is allowed to return 1 (success) even if the J> callout is still currently running; a.k.a. it is not because you J> successfully cancelled a callout that the callout is not currently running. J> J> We did propose a patch to make _callout_stop_safe() returns 0 (fail) J> when the callout is currently running: J> J> callout_stop() should return 0 when the callout is currently being J> serviced and indeed unstoppable J> https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most J> J> But this change impacted too many old code paths and was interesting J> only for TCP timers and thus was abandoned. The fix I am working on now is doing exactly that. callout_reset must return 0 if the callout is currently running. What are the old paths impacted? -- Totus tuus, Glebius. From owner-freebsd-net@freebsd.org Mon Jun 20 10:10:44 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF6B7A7B123 for ; Mon, 20 Jun 2016 10:10:44 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id AC5DC2CD2 for ; Mon, 20 Jun 2016 10:10:44 +0000 (UTC) (envelope-from hps@selasky.org) Received: by mailman.ysv.freebsd.org (Postfix) id AB827A7B120; Mon, 20 Jun 2016 10:10:44 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB08CA7B11F; Mon, 20 Jun 2016 10:10:44 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 72C332CD1; Mon, 20 Jun 2016 10:10:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 0FCB81FE023; Mon, 20 Jun 2016 12:10:40 +0200 (CEST) Subject: Re: panic with tcp timers To: Gleb Smirnoff , Julien Charbon References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> Cc: rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Hans Petter Selasky Message-ID: Date: Mon, 20 Jun 2016 12:14:18 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20160620095822.GJ1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:10:44 -0000 On 06/20/16 11:58, Gleb Smirnoff wrote: > The fix I am working on now is doing exactly that. callout_reset must > return 0 if the callout is currently running. > > What are the old paths impacted? Hi, I'll dig into the matter aswell and give some comments. Thanks for the analysis, Gleb. FYI: This class of problems wouldn't exist if the callout could be associated with a mutex! --HPS From owner-freebsd-net@freebsd.org Mon Jun 20 10:30:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16351A7B538 for ; Mon, 20 Jun 2016 10:30:18 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0095F1845 for ; Mon, 20 Jun 2016 10:30:18 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id ED4F2A7B535; Mon, 20 Jun 2016 10:30:17 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC966A7B533; Mon, 20 Jun 2016 10:30:17 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A8701840; Mon, 20 Jun 2016 10:30:17 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id u5KAUF67094805 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 20 Jun 2016 03:30:15 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id u5KAUFOX094804; Mon, 20 Jun 2016 03:30:15 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 20 Jun 2016 03:30:15 -0700 From: Gleb Smirnoff To: Hans Petter Selasky Cc: Julien Charbon , rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Subject: Re: panic with tcp timers Message-ID: <20160620103015.GK1076@FreeBSD.org> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:30:18 -0000 On Mon, Jun 20, 2016 at 12:14:18PM +0200, Hans Petter Selasky wrote: H> On 06/20/16 11:58, Gleb Smirnoff wrote: H> > The fix I am working on now is doing exactly that. callout_reset must H> > return 0 if the callout is currently running. H> > H> > What are the old paths impacted? H> H> Hi, H> H> I'll dig into the matter aswell and give some comments. Thanks for the H> analysis, Gleb. H> H> FYI: This class of problems wouldn't exist if the callout could be H> associated with a mutex! Exactly! I am convinced that all callouts should be locked, and non-locked one should simply go away, as well as async drain. What does prevent us from converting TCP timeouts to locked? To my understanding it is the lock order of taking pcbinfo after pcb lock. I'm now trying to understand Julien's conversion from pcbinfo lock to pcbinfo + pcblist locks. It seems to me that we actually can convert TCP timers to locked callouts. What for do we need pcbinfo read lock in a tcp timer? The timer works only on the pcb and doesn't modify global lists, does it? -- Totus tuus, Glebius. From owner-freebsd-net@freebsd.org Mon Jun 20 10:45:43 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85C50A7B7E1 for ; Mon, 20 Jun 2016 10:45:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 721F51FA4 for ; Mon, 20 Jun 2016 10:45:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: by mailman.ysv.freebsd.org (Postfix) id 66945A7B7DF; Mon, 20 Jun 2016 10:45:43 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65E4CA7B7DD; Mon, 20 Jun 2016 10:45:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 295551F9D; Mon, 20 Jun 2016 10:45:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4FC1F1FE023; Mon, 20 Jun 2016 12:45:40 +0200 (CEST) Subject: Re: panic with tcp timers To: Gleb Smirnoff , Julien Charbon References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> Cc: rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Hans Petter Selasky Message-ID: <4b5d36b0-a20a-8a1b-b346-8761fbc2e9cf@selasky.org> Date: Mon, 20 Jun 2016 12:49:17 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20160620095822.GJ1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:45:43 -0000 On 06/20/16 11:58, Gleb Smirnoff wrote: > J> callout_stop() should return 0 when the callout is currently being > J> serviced and indeed unstoppable > J> https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most > > What are the old paths impacted? > Hi Gleb, Digging through my e-mail archive rephrasing myself and comments about: https://reviews.freebsd.org/D3078 There are two kinds of callouts. So-called MPSAFE callouts which doesn't have a mutex associated with it, and non-MPSAFE which do. When you are associating a mutex with a callout, callout_stop() will _always_ cancel the callback even if it is pending, and this should be reflected by the return value. Your proposed changes will break that ??? The changes in D3078 didn't take into account "use_lock" at least, and so the return values for the non-MPSAFE case becomes incorrect. > Hi, > > I think this patch is incomplete and can break the return value for non-MPSAFE callouts. I think the correct statement should check the value of "use_lock" too: > > not_running = !(cc_exec_curr(cc, direct) == c && use_lock == 0); > > Because if the callback process waits for lock "c_lock" in the callback process then we have above "cc_exec_curr(cc, direct) == c" is satisfied too, and non-MPSAFE callouts are always cancelable, via cc_exec_cancel(cc, direct) = true; --HPS From owner-freebsd-net@freebsd.org Mon Jun 20 10:48:25 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 241A0A7B8E5 for ; Mon, 20 Jun 2016 10:48:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0C70D21E2 for ; Mon, 20 Jun 2016 10:48:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 07DB5A7B8E3; Mon, 20 Jun 2016 10:48:25 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07679A7B8E2; Mon, 20 Jun 2016 10:48:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A479921DF; Mon, 20 Jun 2016 10:48:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u5KAmJon046864 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 20 Jun 2016 13:48:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u5KAmJon046864 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u5KAmJjv046863; Mon, 20 Jun 2016 13:48:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 20 Jun 2016 13:48:19 +0300 From: Konstantin Belousov To: Julien Charbon Cc: Gleb Smirnoff , rrs@FreeBSD.org, current@FreeBSD.org, hselasky@FreeBSD.org, net@FreeBSD.org Subject: Re: panic with tcp timers Message-ID: <20160620104819.GV38613@kib.kiev.ua> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:48:25 -0000 On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote: > > Hi, > > On 6/20/16 9:39 AM, Gleb Smirnoff wrote: > > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: > > J> > Comparing stable/10 and head, I see two changes that could > > J> > affect that: > > J> > > > J> > - callout_async_drain > > J> > - switch to READ lock for inp info in tcp timers > > J> > > > J> > That's why you are in To, Julien and Hans :) > > J> > > > J> > We continue investigating, and I will keep you updated. > > J> > However, any help is welcome. I can share cores. > > > > Now, spending some time with cores and adding a bunch of > > extra CTRs, I have a sequence of events that lead to the > > panic. In short, the bug is in the callout system. It seems > > to be not relevant to the callout_async_drain, at least for > > now. The transition to READ lock unmasked the problem, that's > > why NetflixBSD 10 doesn't panic. > > > > The panic requires heavy contention on the TCP info lock. > > > > [CPU 1] the callout fires, tcp_timer_keep entered > > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); > > [CPU 2] schedules the callout > > [CPU 2] tcp_discardcb called > > [CPU 2] callout successfully canceled > > [CPU 2] tcpcb freed > > [CPU 1] unblocks... panic > > > > When the lock was WLOCK, all contenders were resumed in a > > sequence they came to the lock. Now, that they are readers, > > once the lock is released, readers are resumed in a "random" > > order, and this allows tcp_discardcb to go before the old > > running callout, and this unmasks the panic. > > Highly interesting. I should be able to reproduce that (will be useful > for testing the corresponding fix). > > Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1 > (success) here) when the callout cancellation is a success _but_ the > callout is current running, that should fix it. > > For the history: It comes back to my old callout question: > > Does _callout_stop_safe() is allowed to return 1 (success) even if the > callout is still currently running; a.k.a. it is not because you > successfully cancelled a callout that the callout is not currently running. > > We did propose a patch to make _callout_stop_safe() returns 0 (fail) > when the callout is currently running: > > callout_stop() should return 0 when the callout is currently being > serviced and indeed unstoppable > https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most > > But this change impacted too many old code paths and was interesting > only for TCP timers and thus was abandoned. Look at callout_stop CS_MIGRBLOCK flag and the fix in sleepq_check_timeout(). Or, at least, do not allow this use of callout_stop() to rot again, after previous dozen regressions and fixes there. From owner-freebsd-net@freebsd.org Mon Jun 20 10:54:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8075A7BB34 for ; Mon, 20 Jun 2016 10:54:00 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 942F727E0 for ; Mon, 20 Jun 2016 10:54:00 +0000 (UTC) (envelope-from hps@selasky.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8D461A7BB32; Mon, 20 Jun 2016 10:54:00 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A4EBA7BB30; Mon, 20 Jun 2016 10:54:00 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4714927DB; Mon, 20 Jun 2016 10:53:59 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id B98351FE023; Mon, 20 Jun 2016 12:53:57 +0200 (CEST) Subject: Re: panic with tcp timers To: Gleb Smirnoff References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <20160620103015.GK1076@FreeBSD.org> Cc: Julien Charbon , rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Hans Petter Selasky Message-ID: Date: Mon, 20 Jun 2016 12:57:35 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20160620103015.GK1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:54:00 -0000 On 06/20/16 12:30, Gleb Smirnoff wrote: > Exactly! I am convinced that all callouts should be locked, and non-locked > one should simply go away, as well as async drain. I agree about that that, except you still need the async drain, because it will prevent freeing the lock protecting the callout, which may still be in use after callout_stop(). --HPS From owner-freebsd-net@freebsd.org Mon Jun 20 10:56:42 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67E9EA7BC43 for ; Mon, 20 Jun 2016 10:56:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 540972AF1 for ; Mon, 20 Jun 2016 10:56:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: by mailman.ysv.freebsd.org (Postfix) id 500F0A7BC41; Mon, 20 Jun 2016 10:56:42 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FA21A7BC40; Mon, 20 Jun 2016 10:56:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16E992AF0; Mon, 20 Jun 2016 10:56:41 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 9270C1FE023; Mon, 20 Jun 2016 12:56:39 +0200 (CEST) Subject: Re: panic with tcp timers To: Gleb Smirnoff References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <20160620103015.GK1076@FreeBSD.org> Cc: Julien Charbon , rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Hans Petter Selasky Message-ID: Date: Mon, 20 Jun 2016 13:00:16 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20160620103015.GK1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 10:56:42 -0000 On 06/20/16 12:30, Gleb Smirnoff wrote: > What does prevent us from converting TCP timeouts to locked? To my > understanding it is the lock order of taking pcbinfo after pcb lock. I started this work: https://reviews.freebsd.org/D1563 --HPS From owner-freebsd-net@freebsd.org Mon Jun 20 11:38:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30541A7B978 for ; Mon, 20 Jun 2016 11:38:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 20A6522BF for ; Mon, 20 Jun 2016 11:38:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5KBc2wJ052935 for ; Mon, 20 Jun 2016 11:38:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Mon, 20 Jun 2016 11:38:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 11:38:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #7 from Mark Johnston --- (In reply to Andrey V. Elsukov from comment #6) Thanks. I think this is a regression from r292978. Before that change, entries crea= ted by nd6_cache_lladdr() without a specified lladdr would have reported an add= ress consisting of zeroes; now we leave ll_addr set to NULL, which causes the problem. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 20 14:18:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95407A78A48 for ; Mon, 20 Jun 2016 14:18:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 73B7214B0 for ; Mon, 20 Jun 2016 14:18:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6EF77A78A45; Mon, 20 Jun 2016 14:18:04 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E7BBA78A44; Mon, 20 Jun 2016 14:18:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36E3814AF; Mon, 20 Jun 2016 14:18:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x22c.google.com with SMTP id f30so122924442ioj.2; Mon, 20 Jun 2016 07:18:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=aSTFxSQHnqXqp0ZxEqAO17TAb9PpagTVJzuXiV0xEbw=; b=HIezpgehjete5ldt32UpVFBjCTzAgoRLWELIb3UD8KY7GHjgtQxb4SVDDe0Mb34uSR YphWiQ2OQ3O3EGOlojwP9Kfu/XGrAAuJLwvjoZKB73leOh+v5/WWpcCutyHguj355DRb IpdrjC+L55QiWwG2i40fXvwKd/KcNw81gHhRXz2OIhm3iCe23NqQud9m6PuzfBuS0VC4 Zht6czzsXZzOe4C4TWGMnNC5wvagq1ps2uUxt3iJoHtqctumc1siZurb6ro3f5m81KeU uYd09prH7xNZRabe0ZZu31Jl80IHfR7xnax+1pcfqoJDyHVU5Ks/xBPTWZ8KQ+uqSxHL wmmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=aSTFxSQHnqXqp0ZxEqAO17TAb9PpagTVJzuXiV0xEbw=; b=e5hMQYebCDIjvIsTQGvrfT4LhBtK8yKT9iEolyq6CfVpyOs5cv/3j38uQJmnddcJ0g Vx7uicFKf16C+DJJES21lIh27XYNypdfLCScd7oaW2IJC6XzWR9ry2gPk6ScjBrEZBU0 NUuZCXosXiu8AgzXAkGFX4/FbQJjroqLehJMAHegvCZgmmW0OYET/II7GO7cmT1IBRVU fovKK6ZZsVjcVUksVN4VWq1JhaWRy3HTEfHWlvN8O9YkyLDpPwiK+dJmYOdDMHvOC+ho b5AIQs7Olk/JG+yvp/qVHAYko0GG04Z86k+rJN1QbHnE/3FDp3RjZUUTn/vQ0nSbpQqy Yh3g== X-Gm-Message-State: ALyK8tLjRSirQE9Z5+jC4A+S6tXF0ez2+/tB68AHtNZV034W/p3Zvn97ZcNNQ35Tv530f/7E3biRgX1+X2zWDg== X-Received: by 10.107.37.19 with SMTP id l19mr23129896iol.75.1466432282575; Mon, 20 Jun 2016 07:18:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.210.212 with HTTP; Mon, 20 Jun 2016 07:18:01 -0700 (PDT) In-Reply-To: References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <20160620103015.GK1076@FreeBSD.org> From: Adrian Chadd Date: Mon, 20 Jun 2016 07:18:01 -0700 Message-ID: Subject: Re: panic with tcp timers To: Hans Petter Selasky Cc: Gleb Smirnoff , Julien Charbon , "net@freebsd.org" , "current@freebsd.org" , Randall Stewart Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 14:18:04 -0000 There's implications for RSS with how the callout system currently works. If you don't know the RSS bucket ID of a connection in advance, you'll create callouts on the wrong CPUs and then they're not migrated. The initial work here did convert things over, but didn't place the callouts in the right CPU/RSS bucket and there wasn't a mechanism to fix this. :( (But I'm also a firm believer of that too. I'd also finally just like the callout system to not be tied to per-CPU queues, but also per-RSS-bucket callout wheels so we could assign a CPU mask to a given callout wheel and then migrating things around is just "change cpu mask", not "change callout cpu id.") -adrian On 20 June 2016 at 04:00, Hans Petter Selasky wrote: > On 06/20/16 12:30, Gleb Smirnoff wrote: >> >> What does prevent us from converting TCP timeouts to locked? To my >> understanding it is the lock order of taking pcbinfo after pcb lock. > > > I started this work: > > https://reviews.freebsd.org/D1563 > > --HPS > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Mon Jun 20 16:37:28 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B256EA7BDD4 for ; Mon, 20 Jun 2016 16:37:28 +0000 (UTC) (envelope-from vic@tjk-source.com) Received: from m13-231.163.com (m13-231.163.com [220.181.13.231]) by mx1.freebsd.org (Postfix) with ESMTP id 1CBA02BB6 for ; Mon, 20 Jun 2016 16:37:22 +0000 (UTC) (envelope-from vic@tjk-source.com) Received: from LocalHost (unknown [114.215.204.92]) by smtp3 (Coremail) with SMTP id PdOowECZR0t1GmhXvfvNBg--.7961S2; Tue, 21 Jun 2016 00:31:50 +0800 (CST) Message-ID: <10547713177382B6C4ACC8C8F8BB0415BF6F9078E9@TJK-SOURCE.COM> Date: Tue, 21 Jun 2016 0:00:09 +0800 From: "Vic Li" Reply-To: To: Subject: caustic soda supplier MIME-Version: 1.0 X-Priority: 3 X-Mailer: Joinf MailSystem 8.0 X-CM-TRANSID: PdOowECZR0t1GmhXvfvNBg--.7961S2 X-Coremail-Antispam: 1Uf129KBjDUn29KB7ZKAUJUUUTP529EdanIXcx71UUUUU7v73 VFW2AGmfu7bjvjm3AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvjxUSE_EUUUUU X-Originating-IP: [114.215.204.92] X-CM-SenderInfo: hylfq31mnn20pxufvhhfrp/1tbiQBeOiFbdGKDCyQAAsp Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 16:37:28 -0000 DQpIaTpzaXINCiANCk5pY2UgZGF5IHRvZGF5DQpUaGlzIGlzIFZpYyBMaSBmb3JtIFRpYW5KaW4g ay1zb3VyY2UgQ28sLkx0ZA0KV2UgaGF2ZSBhIHB1cmUgY29sb3IgQXJlIG9mIGdvb2QgcXVhbGl0 eSBJbXB1cml0aWVzIGluIHRoZSBjYXVzdGljIHNvZGEgcGVhcmxzIGFuZCBjYXVzdGljIHNvZGEg Zmxha2UgQ29tcGFyZWQgdG8gb3RoZXIgcHJvZHVjdHMgb2Ygb3VyIHByb2R1Y3QgY29sb3IgbW9y ZSBwdXJlIHByaWNlIG1vcmUgYXBwcm9wcmlhdGUgDQpMb29rIGZvcndhcmQgdG8gY29vcGVyYXRp b24gd2l0aCB5b3UgDQpSZ2RzDQogDQogdmljIExpDQogVGlhbmppbiBLLXNvdXJjZSBQZXRyb2No ZW1pY2FsIENvLixMdGQuIA0KIEVtYWls77yadmljQHRqay1zb3VyY2XCt2NvbQ0KIE1vYjo4Ni0x NTgyMjcwNzcyMA0KIFRFTDo4Ni0wMjItNjU2MTc3MjANCiBTa3lwZTp2aWNAdGprLXNvdXJjZcK3 Y29tDQogV2ViOnd3dy5rc291cmNlcGV0cm9jaGVtaWNhbC5lbi5hbGliYWJhLmNvbQ0KIEFkZDpS b29tMjUxMSwgemhlbmdzaGFuZyBCdWlsZGluZywgVGFuZ2d1IERpc3RyaWN0LFRpYW5qaW4sQ2hp bmE= From owner-freebsd-net@freebsd.org Mon Jun 20 16:48:50 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4545AC40DE for ; Mon, 20 Jun 2016 16:48:50 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id AD57A1314 for ; Mon, 20 Jun 2016 16:48:50 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id A6672AC40DC; Mon, 20 Jun 2016 16:48:50 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A58F2AC40DA; Mon, 20 Jun 2016 16:48:50 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E2621310; Mon, 20 Jun 2016 16:48:50 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f48.google.com with SMTP id r201so70242096wme.1; Mon, 20 Jun 2016 09:48:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=dspSP47DsoxtDUjUNH0mSVqm+HvYa1GNboskfxte0EI=; b=Btp3NJySvqzKM1XTn8aYcF+knIsTgIf5GIfUWNchIs9Sw3VCFoM/9OcXODBB0/ZHRA lNJLIZQzZ65/FQ+6r3auXjP0OLo4Y21BAnvC9JmTlNAm4sUwqUgpfDIyKuXj3RAGPS3F oPOuKJAJK2NuO4cQ5WLPW8L1b0NM5hSFMiwv4m/jVBfhADpit8VN6s4j43tQ4wvWtomY ibBkIsSudPkn0pFYvxQ++ILWFIMHPdloq62Dp3ymkSejy3vz2XYqdIfX3j6mekNtQvcs iH+mjza6imYON6Uagk0JYEXv44WNaQE6yai4at+f8sJ7UuV72vkTxT1yvllMaO99LY8W D/dw== X-Gm-Message-State: ALyK8tLSWFTWeBZE7oeQ0pNiZYcWHwnx11ar+huVAIQI15PYm5H+Toku8Y+qp5P7wpuB/Q== X-Received: by 10.28.45.201 with SMTP id t192mr12821342wmt.77.1466441317441; Mon, 20 Jun 2016 09:48:37 -0700 (PDT) Received: from [172.20.10.4] (48.228.197.178.dynamic.wless.zhbmb00p-cgnat.res.cust.swisscom.ch. [178.197.228.48]) by smtp.gmail.com with ESMTPSA id t198sm14394128wmt.16.2016.06.20.09.48.36 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 20 Jun 2016 09:48:36 -0700 (PDT) Subject: Re: panic with tcp timers To: Gleb Smirnoff , Hans Petter Selasky References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <20160620103015.GK1076@FreeBSD.org> Cc: rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Julien Charbon Message-ID: <7294457d-07c0-2d6e-617b-15fa48bf2bb9@freebsd.org> Date: Mon, 20 Jun 2016 18:48:27 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160620103015.GK1076@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1Ta12WsRFnONHaUgmQbLo6tFj62n4m1rc" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 16:48:50 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --1Ta12WsRFnONHaUgmQbLo6tFj62n4m1rc Content-Type: multipart/mixed; boundary="Mvq9GL89TXjDlopGgmEC0W8JMOWLNG1QO" From: Julien Charbon To: Gleb Smirnoff , Hans Petter Selasky Cc: rrs@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Message-ID: <7294457d-07c0-2d6e-617b-15fa48bf2bb9@freebsd.org> Subject: Re: panic with tcp timers References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <20160620103015.GK1076@FreeBSD.org> In-Reply-To: <20160620103015.GK1076@FreeBSD.org> --Mvq9GL89TXjDlopGgmEC0W8JMOWLNG1QO Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 6/20/16 12:30 PM, Gleb Smirnoff wrote: > On Mon, Jun 20, 2016 at 12:14:18PM +0200, Hans Petter Selasky wrote: > H> On 06/20/16 11:58, Gleb Smirnoff wrote: > H> > The fix I am working on now is doing exactly that. callout_reset m= ust > H> > return 0 if the callout is currently running. > H> > > H> > What are the old paths impacted? > H>=20 > H> I'll dig into the matter aswell and give some comments. Thanks for t= he=20 > H> analysis, Gleb. > H>=20 > H> FYI: This class of problems wouldn't exist if the callout could be=20 > H> associated with a mutex! >=20 > Exactly! I am convinced that all callouts should be locked, and non-loc= ked > one should simply go away, as well as async drain. >=20 > What does prevent us from converting TCP timeouts to locked? To my > understanding it is the lock order of taking pcbinfo after pcb lock. >=20 > I'm now trying to understand Julien's conversion from pcbinfo lock > to pcbinfo + pcblist locks. It seems to me that we actually can convert= > TCP timers to locked callouts. >=20 > What for do we need pcbinfo read lock in a tcp timer? The timer works > only on the pcb and doesn't modify global lists, does it? tcp_timer_keep() for example can modify global pcb list, see the call stack below: tcp_timer_keep() tcp_drop() tcp_close() sofree() tcp_usr_detach() (via pr->pr_usrreqs->pru_detach() in sofree()) tcp_detach() in_pcbfree() in_pcbremlists() Anyway, a bit of history here: o In stable/10 the TCP locking order is: ipi_lock (before) inpcb lock and in stable/10 ipi_lock is protecting the global pcb list. Then, only in the cases where you were absolutely sure you are _not_ going to modify the global pcb list you are allowed to take the inpcb lock only. For all the other cases, you have to take the write lock on ipi_lock first due to lock order. And in context of short-lived TCP connection of the 5 received packets for a TCP connection, 4 require the write ipi_lock lock, and that's does not scale well. Lesson learned: For scaling perspective, in lock ordering always put the most global lock last. It was improved in a lean way in 11: o In 11 the TCP lock order became: ipi_lock (before) inpcb lock (before) ipi_list And in 11 ipi_list protects global pcb list, and only the code actually modifying the global list is protected by a global write lock, e.g.: https://github.com/freebsd/freebsd/blob/master/sys/netinet/in_pcb.c#L1285= Then why keeping the ipi_lock lock at all now? It is solely for one case: When you need the complete stability of the full pcb list while traversing it. These full pcb list traversals are done in functions like: in_pcbnotifyall(), in_pcbpurgeif0(), inp_apply_all(), tcp_ccalgounload(), tcp_drain(), etc. Thus in 11 ipi_lock write lock is required only when you do this full traversal with list stability requirement, and the ipi_lock read lock in all other cases like tcp_input() that then scales better. Sadly in 11, you cannot use the inpcb lock as is for the TCP timers callout, because like tcp_timer_keep() most TCP timers callout can modify the global pcb list and then you need the read lock ipi_lock in top of inpcb lock... o Future/12: The next natural step is to remove the ipi_lock from the picture to get:= inpcb lock (before) ipi_list It /just/ requires a global pcb list that allows addition/deletion while doing a full traversal concurrently. A classical way to achieve that, is to use a RCU-based list. As soon as RCU (or anything equivalent like lwref) are supported in kernel, we will implement this change. Just history here, as I already did a presentation on this exact topic at BSDCan 2015: https://wiki.freebsd.org/201506DevSummit#line-83 It was recorded but I never saw the footage/presentation actually published. :) -- Julien --Mvq9GL89TXjDlopGgmEC0W8JMOWLNG1QO-- --1Ta12WsRFnONHaUgmQbLo6tFj62n4m1rc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXaB5jAAoJEKVlQ5Je6dhxuN8IAKeytbxsVb7bEK4iGl3HDY8X RRD4rBqamc4cVTzzLn75LpOkIXQZhj6SzaZKO5AfTjZ5eUSuabYubM3wcpai9swx UEoBCnjocNWjxJcElRwdN0CNgj0VtGPlNycWe+d8p3hRGqy14IdGEQ74+ru7ryC8 uGg/eJ0e73YmjVOZCthHHqyH1K+JLild16ZSH+1uFJt+KaPJjLBg5iQfXvM7t1HM yP7HvACAaQXFrUdw/L51pffdxOAwG1FHieQMw2Lu8GuoCn7dJ4hlxumETJpcPJY9 Torn0bCwg0fdSLQiHJbGLE5VywXzcIdb0KAxLBO+k6leCKbnU9lEkxkp7uihB6U= =xbUt -----END PGP SIGNATURE----- --1Ta12WsRFnONHaUgmQbLo6tFj62n4m1rc-- From owner-freebsd-net@freebsd.org Mon Jun 20 17:46:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 677C9A7A15B for ; Mon, 20 Jun 2016 17:46:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4B1511814 for ; Mon, 20 Jun 2016 17:46:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 43234A7A158; Mon, 20 Jun 2016 17:46:04 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42807A7A156; Mon, 20 Jun 2016 17:46:04 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9A50180F; Mon, 20 Jun 2016 17:46:03 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f49.google.com with SMTP id a66so89228845wme.0; Mon, 20 Jun 2016 10:46:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=I5fID0TamFhGqwFKXXP2M0uoJBITkuSZcBf7+qEgfMo=; b=BcGz9FqHx9fnP5e+VxXgPXos0tfIb7tYpHJ4OOqi0IohnGCjS9fN+tEOC/5mJy9K2J Ynwlp6ssMdoU+vgbEWeKlOQtsAkJuGBDeoEIZ64u+m7rdZ6BFoaGJavC+nlSyIfJyRqM fWQz6Vc1i8DxRWnR7WeqD1T3SeYrFhndN+DsNTpybm1ubxePTsJINW03yEi+qGXvdWRZ AX1OjIaeFZH6Jxe4tHr9trxWO+1Mb+6wftSEQgP+ORNLp/mqaatL5sAQ3mzQ1lMETpFJ 0HPhat8ZdV8TJE46BSRdXFPfu9i8oiLk0X1qq5H6VO8/p0TaqOrGzXZ8033OVq32EX9M I6XA== X-Gm-Message-State: ALyK8tKA1HntFiEpMIbttz1Rqb5rnlFTZzEIQ/rCtdWXIFCDkaVedxFKvrxLMatHruyq7Q== X-Received: by 10.28.157.1 with SMTP id g1mr12886852wme.34.1466444755441; Mon, 20 Jun 2016 10:45:55 -0700 (PDT) Received: from [10.100.64.16] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id li10sm20146765wjb.5.2016.06.20.10.45.54 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 20 Jun 2016 10:45:54 -0700 (PDT) Subject: Re: panic with tcp timers To: Gleb Smirnoff References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> Cc: rrs@FreeBSD.org, hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Julien Charbon Message-ID: <74bb31b7-a9f5-3d0c-eea0-681872e6f09b@freebsd.org> Date: Mon, 20 Jun 2016 19:45:53 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160620095822.GJ1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 17:46:04 -0000 Hi, On 6/20/16 11:58 AM, Gleb Smirnoff wrote: > On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote: > J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: > J> > J> > Comparing stable/10 and head, I see two changes that could > J> > J> > affect that: > J> > J> > > J> > J> > - callout_async_drain > J> > J> > - switch to READ lock for inp info in tcp timers > J> > J> > > J> > J> > That's why you are in To, Julien and Hans :) > J> > J> > > J> > J> > We continue investigating, and I will keep you updated. > J> > J> > However, any help is welcome. I can share cores. > J> > > J> > Now, spending some time with cores and adding a bunch of > J> > extra CTRs, I have a sequence of events that lead to the > J> > panic. In short, the bug is in the callout system. It seems > J> > to be not relevant to the callout_async_drain, at least for > J> > now. The transition to READ lock unmasked the problem, that's > J> > why NetflixBSD 10 doesn't panic. > J> > > J> > The panic requires heavy contention on the TCP info lock. > J> > > J> > [CPU 1] the callout fires, tcp_timer_keep entered > J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); > J> > [CPU 2] schedules the callout > J> > [CPU 2] tcp_discardcb called > J> > [CPU 2] callout successfully canceled > J> > [CPU 2] tcpcb freed > J> > [CPU 1] unblocks... panic > J> > > J> > When the lock was WLOCK, all contenders were resumed in a > J> > sequence they came to the lock. Now, that they are readers, > J> > once the lock is released, readers are resumed in a "random" > J> > order, and this allows tcp_discardcb to go before the old > J> > running callout, and this unmasks the panic. > J> > J> Highly interesting. I should be able to reproduce that (will be useful > J> for testing the corresponding fix). > J> > J> Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1 > J> (success) here) when the callout cancellation is a success _but_ the > J> callout is current running, that should fix it. > J> > J> For the history: It comes back to my old callout question: > J> > J> Does _callout_stop_safe() is allowed to return 1 (success) even if the > J> callout is still currently running; a.k.a. it is not because you > J> successfully cancelled a callout that the callout is not currently running. > J> > J> We did propose a patch to make _callout_stop_safe() returns 0 (fail) > J> when the callout is currently running: > J> > J> callout_stop() should return 0 when the callout is currently being > J> serviced and indeed unstoppable > J> https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most > J> > J> But this change impacted too many old code paths and was interesting > J> only for TCP timers and thus was abandoned. > > The fix I am working on now is doing exactly that. callout_reset must > return 0 if the callout is currently running. > > What are the old paths impacted? Actually all the paths that check the callout_stop() return value AND call both callout_reset() and callout_stop() AND use mpsafe callout(). And for each path, we would have to check our patch was ok (or not). Thus, if you only do the change in callout_async_drain() context, you don't impact the "old" callout_stop() behavior and get the desired behavior for the TCP timers. Might be a good trade-off... My 2 cents. -- Julien From owner-freebsd-net@freebsd.org Tue Jun 21 00:34:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A62DDA7A4CE for ; Tue, 21 Jun 2016 00:34:14 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 0CB731AB0 for ; Tue, 21 Jun 2016 00:34:13 +0000 (UTC) (envelope-from freebsd@grem.de) Received: (qmail 54158 invoked by uid 89); 21 Jun 2016 00:27:25 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@194.97.158.70) by mail.grem.de with ESMTPA; 21 Jun 2016 00:27:25 -0000 Date: Tue, 21 Jun 2016 02:27:23 +0200 From: Michael Gmelin To: freebsd-net@freebsd.org Subject: ARP table entries / ifconfig needs to be issued twice when moving IP Message-ID: <20160621022723.5e785573@bsd64.grem.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 00:34:14 -0000 Hi, I'm not sure if it's just me being tired, but I'm facing the following problem on 10.3-RELEASE when moving an IPv4 alias from one host to the other. This is an example of what I'm seeing: Configuration: Box 1: ifconfig_bge0_name="uplink" ifconfig_uplink="inet 10.1.1.253/24 description 'uplink'" ifconfig_uplink_ipv6="inet6 fd8e:1111:1111:1f::fd" mac: 14:18:77:11:22:33 Box 2: ifconfig_bge0_name="uplink" ifconfig_uplink="inet 10.1.1.254/24 description 'uplink'" ifconfig_uplink_ipv6="inet6 fd8e:1111:1111:1f::fe" mac: 14:18:77:44:55:66 Test: Box 1: Configure alias address # ifconfig uplink alias 10.1.1.33/32 # arp -an | grep 10.1.1.33 ? (10.1.1.33) at 14:18:77:11:22:33 on uplink permanent [ethernet] Box 2: Ping, check it's in the ARP table # ping 10.1.1.33 ... # arp -an | grep 10.1.1.33 ? (10.1.1.33) at 14:18:77:11:22:33 on uplink expires in 1188 seconds [ethernet] Box 1: Remove alias # ifconfig uplink -alias 10.1.1.33 Box 2: Add alias # ifconfig uplink alias 10.1.1.33/32 # arp -an | grep 33 ? (10.1.1.33) at 14:18:77:11:22:33 on uplink expires in 1156 seconds [ethernet] It's still in the arp table as a non-permanent entry, pointing to the Box 1. Box 2: Issue ifconfig once more: # ifconfig uplink alias 10.1.1.33/32 # arp -an | grep 33 ? (10.1.1.33) at 14:18:77:44:55:66 on uplink permanent [ethernet] Now it's set to the local ethernet address in the arp table permanently. This is not a big deal in case of a simple setup (packets still flow correctly), but as soon as you add multiple FIBs and interface routes into the mix, things get problematic[1]. Is this intended behaviour, am I missing something? - Michael [1] While the example above was done on two separate boxes, the FIB setup in production is something like this: sysctl net.add_addr_allfibs=0 vlan1: fib 1 vlan2: fib 2 route add -host 10.1.1.33 -interface vlan2 -fib 1 route add -host 10.2.1.32 -interface vlan1 -fib 2 ... -- Michael Gmelin From owner-freebsd-net@freebsd.org Tue Jun 21 00:52:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCEA2A7A857 for ; Tue, 21 Jun 2016 00:52:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCE7D1204 for ; Tue, 21 Jun 2016 00:52:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5L0qIPs025537 for ; Tue, 21 Jun 2016 00:52:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210417] Reproducible igb related panic 11.0-ALPHA4 Date: Tue, 21 Jun 2016 00:52:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 00:52:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210417 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |IntelNetworking Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 21 06:48:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F6E3A7BBC7 for ; Tue, 21 Jun 2016 06:48:09 +0000 (UTC) (envelope-from stdin@niklaas.eu) Received: from box-hlm-03.niklaas.eu (box-hlm-03.niklaas.eu [84.22.110.84]) by mx1.freebsd.org (Postfix) with ESMTP id E297E2DA8 for ; Tue, 21 Jun 2016 06:48:08 +0000 (UTC) (envelope-from stdin@niklaas.eu) Received: by box-hlm-03.niklaas.eu (Postfix, from userid 1001) id 91D3C385519; Tue, 21 Jun 2016 08:48:00 +0200 (CEST) Date: Tue, 21 Jun 2016 08:48:00 +0200 From: Niklaas Baudet von Gersdorff To: freebsd-net@freebsd.org Subject: Re: ARP table entries / ifconfig needs to be issued twice when moving IP Message-ID: <20160621064800.GB8441@box-hlm-03.niklaas.eu> Mail-Followup-To: freebsd-net@freebsd.org References: <20160621022723.5e785573@bsd64.grem.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qtZFehHsKgwS5rPz" Content-Disposition: inline In-Reply-To: <20160621022723.5e785573@bsd64.grem.de> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 06:48:09 -0000 --qtZFehHsKgwS5rPz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Michael Gmelin [2016-06-21 02:27 +0200] : > I'm not sure if it's just me being tired, but I'm facing the following > problem on 10.3-RELEASE when moving an IPv4 alias from one host to > the other. This is an example of what I'm seeing: [...] > It's still in the arp table as a non-permanent entry, pointing to the > Box 1. >=20 > Box 2: Issue ifconfig once more: [...] > Now it's set to the local ethernet address in the arp table > permanently. Reminds me of what I encountered recently with two virtual machines at my provider. I haven't further dug into this, as I didn't check `arp -an`, but what I saw was switching an IP address from hosts A to B resulted in pings from host C to arrive at A. Host C said they got lost while tcpdump showed incoming packets at A although they should have gone to B. Niklaas --qtZFehHsKgwS5rPz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXaOMTAAoJEG2fODeJrIU/jxoP/1z9CBIuv2oyUox4hhspldoV ytee8o2KqVFyhq6U/hir1169iW8GJ5C0DMyHhfStvitfAY9wrkmr2OwmWqRyFE91 ORzSlH3koDKij5g920LXOUxXxZeS+Nn2dzSUvRQe1E1C+1AQh4i9aywDVdIFFJwp vkDuM5d4dx/C5j4U/1RIzKay6O0YThfjJemE/ms+9jqH81GcSK7JnRZt4jetGV9F HbS10JkYdvdNatfL8oBVXZGbsiI9Zqr+SNi9wJjAKvq6hC3y/0T7Wlz38pOakIwV g2VHu70N12D795RL5ZEjKB6QDpHDssYF4eS2JVYQX+riW1etlOvT4Y0caBNVXzBc 1F/yZWwCHZedI/WVQcZqu3SoKM5SeKMtt2IDOax+pAVI1tpsmr8pnz+NNIRMbnta rKi1mbEt38PzgHVdMjl8lZg8uljI6r7nh/P5x4/CKrLofWieQkJ/lQQznXH45jfH PzPs/43yQR9P8ZKEk2vcM4dIpYTQqTuWAxC+Y4ZDuilqBbF2Pp4oZ26guv9cgmnX QWdwkV9fCKQ6y8k0qCKb/PgfFf6proFzxh4KKvR0Mu9/bIj5koJaVy24OhNsnoNv K9LV9mpbW+dpTq2iTgdl1ehL6eeWvjP2RDn/vn+aC2Uuhi7vPuQBxJ9PqZx4tcvR eVWiq8YP5WosB0/GOt5I =muFm -----END PGP SIGNATURE----- --qtZFehHsKgwS5rPz-- From owner-freebsd-net@freebsd.org Tue Jun 21 09:33:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDB8CAC5ECF for ; Tue, 21 Jun 2016 09:33:04 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 501F92753 for ; Tue, 21 Jun 2016 09:33:03 +0000 (UTC) (envelope-from freebsd@grem.de) Received: (qmail 61100 invoked by uid 89); 21 Jun 2016 09:25:40 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@194.97.158.70) by mail.grem.de with ESMTPA; 21 Jun 2016 09:25:40 -0000 Date: Tue, 21 Jun 2016 11:25:38 +0200 From: Michael Gmelin To: freebsd-net@freebsd.org Cc: stdin@niklaas.eu Subject: Re: ARP table entries / ifconfig needs to be issued twice when moving IP Message-ID: <20160621112538.784e7b6e@bsd64.grem.de> In-Reply-To: <20160621064800.GB8441@box-hlm-03.niklaas.eu> References: <20160621064800.GB8441@box-hlm-03.niklaas.eu> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 09:33:05 -0000 > Reminds me of what I encountered recently with two virtual > machines at my provider. I haven't further dug into this, as > I didn't check `arp -an`, but what I saw was switching an IP > address from hosts A to B resulted in pings from host C to arrive > at A. Host C said they got lost while tcpdump showed incoming > packets at A although they should have gone to B. As the packets came from host C, this is expected behaviour. Host C had the ARP entry cached (or if it wasn't on the same network, some network equipment had). What I'm describing is the machine having the IP address configured not creating a permanent entry in the local ARP table for its own interface. - Michael -- Michael Gmelin From owner-freebsd-net@freebsd.org Tue Jun 21 09:37:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE1CBAC5FC2 for ; Tue, 21 Jun 2016 09:37:26 +0000 (UTC) (envelope-from stdin@niklaas.eu) Received: from box-hlm-03.niklaas.eu (box-hlm-03.niklaas.eu [84.22.110.84]) by mx1.freebsd.org (Postfix) with ESMTP id 7C02929E4 for ; Tue, 21 Jun 2016 09:37:26 +0000 (UTC) (envelope-from stdin@niklaas.eu) Received: by box-hlm-03.niklaas.eu (Postfix, from userid 1001) id 4B7EC385519; Tue, 21 Jun 2016 11:37:24 +0200 (CEST) Date: Tue, 21 Jun 2016 11:37:24 +0200 From: Niklaas Baudet von Gersdorff To: freebsd-net@freebsd.org Subject: Re: ARP table entries / ifconfig needs to be issued twice when moving IP Message-ID: <20160621093724.GI8441@box-hlm-03.niklaas.eu> Mail-Followup-To: freebsd-net@freebsd.org References: <20160621064800.GB8441@box-hlm-03.niklaas.eu> <20160621112538.784e7b6e@bsd64.grem.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jIYo0VRlfdMI9fLa" Content-Disposition: inline In-Reply-To: <20160621112538.784e7b6e@bsd64.grem.de> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 09:37:26 -0000 --jIYo0VRlfdMI9fLa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Michael Gmelin [2016-06-21 11:25 +0200] : > As the packets came from host C, this is expected behaviour. Host C had > the ARP entry cached (or if it wasn't on the same network, some network > equipment had). Ok. Thanks a lot for clarifying this. > What I'm describing is the machine having the IP address configured > not creating a permanent entry in the local ARP table for its own > interface. I hope you will get a response from someone who is more knowledgable in this than I am. Niklaas --jIYo0VRlfdMI9fLa Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXaQrNAAoJEG2fODeJrIU/yMkP/ROaE4fTrVD4bf2TrownuvzJ 37Skvv+Tas9P15EPrf26py23Js6dhLMbHMcBTUBQZx3Pg2pd4cBr4DFGMKSeQmvj inW/VbEeCkmluFVaGUYwgkPzgk1KT0aOpUf37zhZI4JU64kL2nSi/s7egRFC9ubT 7mbvChtg4ZREfLI4KFPeRCl9pjyr4b6EhqzkYJlnkvpkD32JY/8ke0HvayybBACp fYzlw/RTu8wxelyMNBwZkwsRKVPQui8UUkJ6EDzWBIIplHzhHgThTHFpLklG3KIT JVVVKuu7TS5V50MJ2wuPK9trFt0XIPTcseuvwewHAboyRFGpjuio07pmyEHLRVTF Wo4eERRZ34u/UaEEGngXtMPE3sla82n3eVeKY8u0MHWo/lOF+OpD9ab5eb0IsTE0 dF9Ngjz6x8fRPrwoRIBoytumviBScD7M/H6+Dixb9RGMmf9K5gZbZXthfV0lC3Pb gKewsYOooJ47orOSkaBO8/Py+VVhVw7sAuLvxJlTJSkyLJrIw40/knW/V9s6qtNX OMM4uxo9+tvJ8guTBT8t6T/0Koeb0YWWlgpvM+KnYpcF5pWklQwipzyggFFI7RGG GHhZ0ebVUJQXSWI07tbZa/riv9oIrY/Ix/lwPoC4RCMXhkJjnXfYc9WLdrhZNz0Z pp1hZKdbkat1gNy147hG =2Es0 -----END PGP SIGNATURE----- --jIYo0VRlfdMI9fLa-- From owner-freebsd-net@freebsd.org Tue Jun 21 11:19:28 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CC48AC4AF9 for ; Tue, 21 Jun 2016 11:19:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7CD7C1F89 for ; Tue, 21 Jun 2016 11:19:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5LBJSQH007177 for ; Tue, 21 Jun 2016 11:19:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210417] Reproducible igb related panic 11.0-ALPHA4 Date: Tue, 21 Jun 2016 11:19:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: arcadiy@ivanovy.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 11:19:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210417 arcadiy@ivanovy.net changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arcadiy@ivanovy.net --- Comment #2 from arcadiy@ivanovy.net --- Bug 200420 might be related without the crash in the 10.x branch. Same adap= ters (I210), motherboard Supermicro X11SBA-F. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 00:24:23 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86A4BAC5894 for ; Wed, 22 Jun 2016 00:24:23 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3897D29AB for ; Wed, 22 Jun 2016 00:24:23 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.ijs.si (Postfix) with ESMTPS id 3rZ51t5m07z1KQ for ; Wed, 22 Jun 2016 02:24:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1466555052; x=1469147053; bh=CY7 fjJn7pBxGVsT7KYCesyeEl8ZptP0eT9owb5R1fJo=; b=AjVFK13nAUf5MoFl4g8 ou9O/2J1VlEr4bqnx8+rwUHpq9uGtLT+zz4CIe0DlA7iuFAlbEiPnTPAKZ90cGYd IU0LglgCH5SWGTUdNSwqIY9i4alZUcfmyp6wdt6V04hJWTvK5B/7TLWVH1H3+9Cy bjvIcaNgOUyUtZY+UAx1pQIY= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id va1bTzFgAD9V for ; Wed, 22 Jun 2016 02:24:12 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3rZ51m31msz1KP for ; Wed, 22 Jun 2016 02:24:12 +0200 (CEST) Received: from nabiralnik.ijs.si (nabiralnik.ijs.si [IPv6:2001:1470:ff80::80:16]) by mildred.ijs.si (Postfix) with ESMTP id 3rZ51m0hwnzXb for ; Wed, 22 Jun 2016 02:24:12 +0200 (CEST) Received: from sleepy.ijs.si (2001:1470:ff80:e001::1:1) by webmail.ijs.si with HTTP (HTTP/1.1 POST); Wed, 22 Jun 2016 02:24:12 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 22 Jun 2016 02:24:12 +0200 From: Mark Martinec To: freebsd-net@freebsd.org Subject: Re: DHCPv6 Support in FreeBSD Base Organization: Jozef Stefan Institute In-Reply-To: <21444224-3DE9-4F5C-AE19-A97DE031D807@DELL.com> References: <6224EC83-3A81-4CE7-83C5-674628F38958@DELL.com> <4a318c6c-ab03-e63c-979f-502bc2afb97e@bluerosetech.com> <21444224-3DE9-4F5C-AE19-A97DE031D807@DELL.com> Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.1.5 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 00:24:23 -0000 On 2016-06-10 23:27, David Bright wrote: > On Jun 10, 2016, at 13:06, Mel Pilgrim > > > wrote: > > Could the WIDE client be used instead? Unlike the ISC client, it will > configure downstream interfaces from PD prefixes without needing an > external script. It also completely avoids the problem of trying to > update the in-base dhclient. > > The ISC client isn’t the only choice. I chose to work with the ISC > client in my work because the current FreeBSD dhclient shares a common > ancestry with the current ISC dhclient and also because the ISC > dhclient implements pseudo interfaces, which is a feature that I need > for my application. > > Is the WIDE client still maintained? When I looked, the most recent > release I found was from 2008. Perhaps you have a link to a more > recent version of the WIDE client. > > Thanks. > David A. Bright The dhcpcd seems like another good choice. http://roy.marples.name/projects/dhcpcd/index It is licensed under the 2 clause BSD license, supports BSD, is being actively developed, is in ports: net/dhcpcd Seems to be the only one to support RFC 7217 (Opaque Interface Identifiers with SLAAC = Stable Private Addresses) which currently lacks support on FreeBSD as far as I can tell (but does work on newer linux). Some of the notable features: IPv6 Router Soliciation including optional address and route management IPv6 Router Advertisement Options for DNS Configuration IPv6 Stable Private Addresses Seamless quad stack of DHCPv4, IPv4LL, IPv6RS, DHCPv6 Small runtime, 200k on amd64 NetBSD ... Mark From owner-freebsd-net@freebsd.org Wed Jun 22 06:37:10 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91F2BAC5EF7 for ; Wed, 22 Jun 2016 06:37:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81F0E1B9A for ; Wed, 22 Jun 2016 06:37:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5M6b9QE020557 for ; Wed, 22 Jun 2016 06:37:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Wed, 22 Jun 2016 06:37:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 06:37:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #8 from Andrey V. Elsukov --- (In reply to Mark Johnston from comment #7) > I think this is a regression from r292978. Before that change, entries > created > by nd6_cache_lladdr() without a specified lladdr would have reported an > address > consisting of zeroes; now we leave ll_addr set to NULL, which causes the > problem. It looks like inet6 code also needs to handle LLE_VALID flag like here https://svnweb.freebsd.org/base/head/sys/netinet/in.c?view=3Dmarkup#l1412 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 06:42:56 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E6BEAC6156 for ; Wed, 22 Jun 2016 06:42:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5EAF71EDD for ; Wed, 22 Jun 2016 06:42:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5M6gtcn037036 for ; Wed, 22 Jun 2016 06:42:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Wed, 22 Jun 2016 06:42:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 06:42:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #9 from Andrey V. Elsukov --- In my coredump lle is for remote address and has only LLE_LINKED flag. So, = to reliable trigger such panic we just need incomplete LLE entry and run `ndp = -a`. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 11:16:59 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F7BEAC5467 for ; Wed, 22 Jun 2016 11:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3CC2276F for ; Wed, 22 Jun 2016 11:16:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MBGww7054712 for ; Wed, 22 Jun 2016 11:16:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 204735] [net] Outgoing packets being sent via wrong interface Date: Wed, 22 Jun 2016 11:16:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ddb@neosystem.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 11:16:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204735 --- Comment #3 from Daniel Bilik --- After moving flowtable out of the picture, the affected router has been run= ning with no problems for more than a month now. So I guess that described probl= em can be definitely considered flowtable-related (with r290276 - still not MF= Ced - being a fix candidate). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 11:17:41 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D09BAC54BD for ; Wed, 22 Jun 2016 11:17:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D1502831 for ; Wed, 22 Jun 2016 11:17:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MBHfX3055662 for ; Wed, 22 Jun 2016 11:17:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 204735] [net] [flowtable] Outgoing packets being sent via wrong interface Date: Wed, 22 Jun 2016 11:17:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ddb@neosystem.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 11:17:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204735 Daniel Bilik changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|[net] Outgoing packets |[net] [flowtable] Outgoing |being sent via wrong |packets being sent via |interface |wrong interface --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 11:29:39 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF2EEAC59FC for ; Wed, 22 Jun 2016 11:29:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF59D116F for ; Wed, 22 Jun 2016 11:29:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MBTdHA077306 for ; Wed, 22 Jun 2016 11:29:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Wed, 22 Jun 2016 11:29:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 11:29:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 --- Comment #10 from commit-hook@freebsd.org --- A commit references this bug: Author: ae Date: Wed Jun 22 11:29:22 UTC 2016 New revision: 302081 URL: https://svnweb.freebsd.org/changeset/base/302081 Log: Fix the NULL pointer dereference for unresolved link layer entries in the netinet6 code. Copy link layer address only when corresponding entry has LLE_VALID flag. PR: 210379 Approved by: re (kib) Changes: head/sys/netinet6/in6.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 11:30:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D615BAC5B00 for ; Wed, 22 Jun 2016 11:30:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C61C2124C for ; Wed, 22 Jun 2016 11:30:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MBUW1c078689 for ; Wed, 22 Jun 2016 11:30:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210379] [panic] in6_lltable_dump_entry bcopy page fault Date: Wed, 22 Jun 2016 11:30:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 11:30:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210379 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 13:00:34 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68157AC5CE4 for ; Wed, 22 Jun 2016 13:00:34 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mx1.freebsd.org (Postfix) with ESMTP id 418CC2672 for ; Wed, 22 Jun 2016 13:00:34 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id D87D111E47; Wed, 22 Jun 2016 13:00:33 +0000 (UTC) Date: Wed, 22 Jun 2016 13:00:33 +0000 To: freebsd-net@freebsd.org From: "bz (Bjoern A. Zeeb)" Reply-to: D1944+325+8925873bdc96dfc2@reviews.freebsd.org Subject: [Differential] D1944: PF and VIMAGE fixes Message-ID: <65873058c8e55a2a2bcd709dbecf3472@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFdqi/E= MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 13:00:34 -0000 YnogYWRkZWQgYSBjb21tZW50LgoKCiAgQ2FuIEkgaGF2ZSB5b3UgZ3V5cyBoYXZlIGEgbG9vayBh dCBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDY5MjQKICAKICBUaGFua3MKClJFVklTSU9O IERFVEFJTAogIGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EMTk0NAoKRU1BSUwgUFJFRkVS RU5DRVMKICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvc2V0dGluZ3MvcGFuZWwvZW1haWxw cmVmZXJlbmNlcy8KClRvOiBudmFzcy1nbXguY29tLCB0cm9jaW55LCBrcmlzdG9mLCBnbm4sIHpl Yywgcm9kcmlnYywgZ2xlYml1cywgZXJpLCBiegpDYzogcnlhbl90aW1ld2FzdGVkLm1lLCBtbW9s bCwgamF2aWVyX292aV95YWhvby5jb20sIGZhcnJva2hpLCBqdWxpYW4sIHJvYmFrLCBmcmVlYnNk LXZpcnR1YWxpemF0aW9uLWxpc3QsIGZyZWVic2QtcGYtbGlzdCwgZnJlZWJzZC1uZXQtbGlzdAo= From owner-freebsd-net@freebsd.org Wed Jun 22 14:27:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0C74B6981A for ; Wed, 22 Jun 2016 14:27:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0ABD1C06 for ; Wed, 22 Jun 2016 14:27:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MERtNV007574 for ; Wed, 22 Jun 2016 14:27:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 209581] igb vf driver does not correctly handle vlan tag Date: Wed, 22 Jun 2016 14:27:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: emaste@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 14:27:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209581 Ed Maste changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |emaste@freebsd.org --- Comment #1 from Ed Maste --- Can you confirm that this is actually running on an arm64 (64-bit ARM) platform, or otherwise update the Hardware description? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 22 14:28:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FC55B698CE for ; Wed, 22 Jun 2016 14:28:13 +0000 (UTC) (envelope-from dpd@dpdtech.com) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE3C41CB1 for ; Wed, 22 Jun 2016 14:28:12 +0000 (UTC) (envelope-from dpd@dpdtech.com) Received: by mail-pa0-x22a.google.com with SMTP id bz2so17454547pad.1 for ; Wed, 22 Jun 2016 07:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dpdtech.com; s=google; h=from:subject:date:message-id:to:mime-version; bh=+NwLZfE47qHoEz8KRbvJCRCmvbP+ElPcl40DuvDM0XU=; b=Fs9qSs0ZBhZs0rfWqxjUMJ1woaJ3mo/T2fH2hkfKIJHyuVvPHz02br8+xloUyILXaP mVXyP6EFRDP7DFOCeBG839diK/dJNszsHKv5VXI4ef0cBdXVJCB8H8KS/GeOpOSVpygu Q9qAmkX0nxZFnbApbtX2ZWSPSvHTfhNyxCIVY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:date:message-id:to:mime-version; bh=+NwLZfE47qHoEz8KRbvJCRCmvbP+ElPcl40DuvDM0XU=; b=U7Xdh7KhqNXTptNIKgna5uI2WVtti0GgVQu9wosJBlMxhCU5SJ6wDhgtaP+HBr3T4S 2tgB7K9rPtNt0RNMjxEd1UNSsiyvzCBda0NOkWOYdSkUP13jL+yW3KGr/UfgFmaaEM3j xU0f/gBcLqNE7zVIQacBoCBfiGucxICAOuoPlkT2M6xqOItw/GW53m/lSph11v161U9M wyJxfCBlY9XLSOTmOhrM7tnyCeLbYcKLZvFjaHkNkkvu/yqqtcuQWdh/DdEGHmJ+db15 1l6pZnIz41e29lcQGblNPigEs80jMqV5mzogMdVHDXI5mszrmmEzZDPx3lk0KOGVmj1Q LUDw== X-Gm-Message-State: ALyK8tLQZ7rnXNmOF0oQ0YlnE1nPJ6F+XAYsBPxr2lorpYhQ74XaFOFm0k1MTp7Af3bl3w== X-Received: by 10.66.151.71 with SMTP id uo7mr35255878pab.134.1466605692089; Wed, 22 Jun 2016 07:28:12 -0700 (PDT) Received: from defiant.dpdtech.com (173-13-188-41-sfba.hfc.comcastbusiness.net. [173.13.188.41]) by smtp.gmail.com with ESMTPSA id g26sm253639pfj.82.2016.06.22.07.28.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 22 Jun 2016 07:28:10 -0700 (PDT) From: "David P. Discher" X-Pgp-Agent: GPGMail 2.6b2 Content-Type: multipart/signed; boundary="Apple-Mail=_66FAE9E9-230A-43A2-8FC0-C6D517D98446"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: Chelsio 10GB PCI-e Opt Card PCI-E 110-1088-30 is a T320 supported via cxgb(4) Date: Wed, 22 Jun 2016 07:28:08 -0700 Message-Id: <2EAD6C36-37AF-4271-AF7B-E1FF1E03FCCE@dpdtech.com> To: FreeBSD Net Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 14:28:13 -0000 --Apple-Mail=_66FAE9E9-230A-43A2-8FC0-C6D517D98446 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 For the Community Documentation on the inter-webs and Google searches - For about a month or two, I was trying to figure out exactly which = platform the Chelsio 10GB Opt Card, part number 110-1088-30 was built = on, and if it was supported under FreeBSD. I suspected it was an N320, but could not confirm it. Chelsio's site = didn=E2=80=99t even have any cross reference for this part number. There are various eBay auctions for these cards, with some PCB variants, = running at the $25-35 range, which would seem to be a steal for a dual = ported 10Gbps ethernet card. So, I broken down and purchased one of these part number 110-1088-30 = PCIe 10Gb =E2=80=9COpt Cards=E2=80=9D. - http://www.ebay.com/itm/351719918339 In fact, this does appears to be the Terminator 3 ASIC platform (T3). I = assume this was later rebranded/rev=E2=80=99ed by Chelsio to the T3 = Unified Wire collection under product name =E2=80=9CN320=E2=80=9D. But = can=E2=80=99t find any references or documentation to confirm this. I don=E2=80=99t know how to probe FreeBSD to check the PCIe sync up. = However, I believe it is a PCIe 1.1 x8 device. This means the PCIe x8 = bus maxes out at 16 Gbps. However, it appears that the T3 version on = this card maxes out at about 11 Gbps (~5.5 Gbps each port with iperf = when lighting up both ports at the same time). This card - even as the N320, the marketing material lists this as a = failover/HA card, not intended for a 20 Gbps LAG. I also found some new, Finisar SFP+ SR optics on eBay for about = $18-20/each. Combined with some fiber from mono price. For about $50, = this feels like a pretty good and cheap solution for cheap 10Gbps = connectivity for home labs/NASes - with a really good and well supported = brand/card. (** This should work in FreeNAS, at least by the kernel, too - the = cxgb(4) support has been around for a long time ! *** ) Hopefully someone at some point down the road, finds this info useful. =3D=3D=3D pciconf -lv =3D=3D=3D cxgbc0@pci0:8:0:0: class=3D0x020000 card=3D0x00011425 = chip=3D0x00311425 rev=3D0x00 hdr=3D0x00 vendor =3D 'Chelsio Communications Inc' device =3D 'T320 10GbE Dual Port Adapter' class =3D network subclass =3D ethernet =3D=3D dmesg, verbose boot =3D=3D=3D pcib8: at device 4.0 on pci0 pcib0: allocated type 3 (0xd8300000-0xd83fffff) for rid 20 of pcib8 pcib8: domain 0 pcib8: secondary bus 8 pcib8: subordinate bus 8 pcib8: memory decode 0xd8300000-0xd83fffff pcib8: special decode ISA pci8: on pcib8 pcib8: allocated bus range (8-8) for rid 0 of pci8 pci8: domain=3D0, physical bus=3D8 found-> vendor=3D0x1425, dev=3D0x0031, revid=3D0x00 domain=3D0, bus=3D8, slot=3D0, func=3D0 class=3D02-00-00, hdrtype=3D0x00, mfdev=3D0 cmdreg=3D0x0147, statreg=3D0x0010, cachelnsz=3D8 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) intpin=3Da, irq=3D7 powerspec 3 supports D0 D3 current D0 MSI supports 32 messages, 64 bit MSI-X supports 32 messages in map 0x20 map[10]: type Memory, range 64, base 0xd8301000, size 12, = enabled pcib8: allocated memory range (0xd8301000-0xd8301fff) for rid 10 of = pci0:8:0:0 map[20]: type Memory, range 64, base 0xd8300000, size 12, = enabled pcib8: allocated memory range (0xd8300000-0xd8300fff) for rid 20 of = pci0:8:0:0 pcib8: matched entry for 8.0.INTA pcib8: slot 0 INTA hardwired to IRQ 16 cxgbc0: mem = 0xd8301000-0xd8301fff,0xd8300000-0xd8300fff irq 16 at device 0.0 on pci8 cxgbc0: attempting to allocate 9 MSI-X vectors (32 supported) msi: routing MSI-X IRQ 258 to local APIC 0 vector 54 msi: routing MSI-X IRQ 259 to local APIC 0 vector 55 msi: routing MSI-X IRQ 260 to local APIC 0 vector 56 msi: routing MSI-X IRQ 261 to local APIC 0 vector 57 msi: routing MSI-X IRQ 262 to local APIC 0 vector 58 msi: routing MSI-X IRQ 263 to local APIC 0 vector 59 msi: routing MSI-X IRQ 264 to local APIC 0 vector 60 msi: routing MSI-X IRQ 265 to local APIC 0 vector 61 msi: routing MSI-X IRQ 266 to local APIC 0 vector 62 cxgbc0: using IRQs 258-266 for MSI-X cxgbc0: using MSI-X interrupts (9 vectors) cxgb0: on cxgbc0 cxgb0: Using defaults for TSO: 65518/35/2048 cxgb0: bpf attached cxgb0: Ethernet address: 00:07:43:0a:a0:84 cxgb1: on cxgbc0 cxgb1: Using defaults for TSO: 65518/35/2048 cxgb1: bpf attached cxgb1: Ethernet address: 00:07:43:0a:a0:85 cxgbc0: Firmware Version 7.11.0 =3D=3D=3D=3D=3D=3D=3D=3D - David P. Discher http://davidpdischer.com/ --Apple-Mail=_66FAE9E9-230A-43A2-8FC0-C6D517D98446 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXaqB5AAoJEEmwU6XuhYWOaW4H/Rlde26B9T4T7dZBw9FHvwpG s8J3IxT8yyh7onilsX7IPZ6e2Lx+cql7B0QbiS+597ZZ8lELbbEmC4grZro5bmMY 2py5xV43dYRd7Pp8yd1gdUHZzVRsF0tQlaOmk4/KkWO/wY4sE4m+W2JwikFX4DXQ uS72r0zdPgDQf6JfFxezfxU8Tr12uAAkn+S+0Awe4kQnV+5sjTb2DUiuGzJLETut VJ0UbREJcDav0ojTCl5TmxOeoV4mogPUr7pr5cxP8EBjdnk6AHBRLVO7PdjiVYYD EPAkqEBE+cd/aYkFqnM5lNe094VFtlGeB4FQ7yAQxO+8KYm+fBiYnCYBbShmDqo= =71Jw -----END PGP SIGNATURE----- --Apple-Mail=_66FAE9E9-230A-43A2-8FC0-C6D517D98446-- From owner-freebsd-net@freebsd.org Wed Jun 22 14:30:22 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88F7CB69AF6 for ; Wed, 22 Jun 2016 14:30:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 789071E8C for ; Wed, 22 Jun 2016 14:30:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5MEUMZW011872 for ; Wed, 22 Jun 2016 14:30:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 209581] igb vf driver does not correctly handle vlan tag Date: Wed, 22 Jun 2016 14:30:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: hlmasterchief93@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: rep_platform Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 14:30:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209581 Long Hoang changed: What |Removed |Added ---------------------------------------------------------------------------- Hardware|arm64 |amd64 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 23 11:08:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF4C1B73241 for ; Thu, 23 Jun 2016 11:08:03 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BD5D2A73 for ; Thu, 23 Jun 2016 11:08:03 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.15.2/8.15.2) with ESMTPSA id u5NArtit054794 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 23 Jun 2016 11:53:57 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Thu, 23 Jun 2016 11:53:47 +0100 From: Karl Pielorz To: freebsd-net@FreeBSD.org Subject: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 11:08:03 -0000 Hi, We're in the process of updating our boxes from 10.1 to 10.3. This has gone OK for the simpler cases - but I seem to have found a couple of issues with the way 10.3 handles both configuring VLANs and actual traffic on VLANs. On our box to be upgraded, our /etc/rc.conf has: cloned_interfaces="lagg0 lagg1 lagg1.30 lagg1.35" ifconfig_bge0="up" ifconfig_bge1="up" ifconfig_lagg0="laggproto failover laggport bge0 laggport bge1 172.16.50.1 netmask 255.255.255.0" ifconfig_em3="mtu 1504 up" ifconfig_em0="mtu 1504 up" ifconfig_lagg1="laggproto failover laggport em3 laggport em0 192.168.0.2 netmask 255.255.255.0 mtu 1504" ifconfig_lagg1_30="inet 192.168.200.2 netmask 255.255.255.0 mtu 1500" ifconfig_lagg1_35="inet 192.168.210.2 netmask 255.255.255.0 mtu 1500" The mtu 'hackery' is needed to avoid MTU issues with VLAN interfaces. The above worked fine under 10.1 - but the same config under 10.3: - Creates lagg0 correctly, and assigns the 172.16.50.1 IP to it - Creates lagg1 - and it's VLAN's - Does not assign 192.168.0.2 to lagg1 (it silently fails to - i.e. no errors logged / shown) So when the system has finished booting you end up with: lagg0 = 172.16.50.1 lagg1 = no IP assigned lagg1.30 = 192.168.200.2 lagg1.35 = 192.168.210.2 The other thing I've found is, once the box is up: #ping 192.168.200.1 PING 192.168.200.1 (192.168.200.1): 56 data bytes ping: sendto: Host is down ^C --- 192.168.200.1 ping statistics --- 6 packets transmitted, 0 packets received, 100.0% packet loss Hmm, not good. 192.168.200.1 is a host on the VLAN 30 network (and is up - I'm logged into it on another session). Same happens for the 192.168.210.0/24 network. Running tcpdump on 192.168.200.1 I see lots of: 11:31:52.956094 ARP, Request who-has 192.168.200.1 tell 192.168.200.2, length 46 11:31:52.956102 ARP, Reply 192.168.200.1 is-at x:x:x:x:x:x, length 28 11:31:53.969140 ARP, Request who-has 192.168.200.1 tell 192.168.200.2, length 46 11:31:53.969148 ARP, Reply 192.168.200.1 is-at x:x:x:x:x:x, length 28 Ok, so the other box can see the ARP requests from the 10.3 box - and issues a reply, but the 10.3 box can't "ping" it. This gets increasingly weird if I run tcpdump on the 10.3 box. The act of running 'tcpdump -i lagg1.30 -n' actually fixes the problem: #ping 192.168.200.1 PING 192.168.100.1 (192.168.200.1): 56 data bytes 64 bytes from 192.168.200.1: icmp_seq=0 ttl=64 time=0.257 ms 64 bytes from 192.168.200.1: icmp_seq=1 ttl=64 time=0.168 ms 64 bytes from 192.168.200.1: icmp_seq=2 ttl=64 time=0.320 ms If I ctrl-c the tcpdump on the 10.3 box at this point - pings stop dead. Restart the tcpdump - pings resume. Restoring 10.1 on the box fixes this - but I'd obviously rather be using 10.3 now. Any ideas? Thanks, -Karl From owner-freebsd-net@freebsd.org Thu Jun 23 12:15:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A813B73A6E for ; Thu, 23 Jun 2016 12:15:05 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C6B61A8C for ; Thu, 23 Jun 2016 12:15:04 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.15.2/8.15.2) with ESMTPSA id u5NCF1hj061381 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 23 Jun 2016 13:15:02 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Thu, 23 Jun 2016 13:14:53 +0100 From: Karl Pielorz To: freebsd-net@FreeBSD.org Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: <2033B3FC769B74294656A089@[10.12.30.106]> In-Reply-To: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 12:15:05 -0000 --On 23 June 2016 11:53 +0100 Karl Pielorz wrote: > This gets increasingly weird if I run tcpdump on the 10.3 box. The act of > running 'tcpdump -i lagg1.30 -n' actually fixes the problem: As a follow up - running 'ifconfig lagg1 promisc' fixes the issue as well (as you'd kind of expect if tcpdump does while it's running). I don't know if that's a good idea / workaround for now? -Kp From owner-freebsd-net@freebsd.org Thu Jun 23 12:32:30 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63341A7A060 for ; Thu, 23 Jun 2016 12:32:30 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E965824BA for ; Thu, 23 Jun 2016 12:32:29 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from ox-dell39.ox.adestra.com (unknown [85.199.232.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 7432D24AF for ; Thu, 23 Jun 2016 12:32:24 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/7432D24AF; dkim=none; dkim-atps=neutral Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? To: freebsd-net@freebsd.org References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> From: Matthew Seaman Message-ID: Date: Thu, 23 Jun 2016 13:32:16 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <2033B3FC769B74294656A089@[10.12.30.106]> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="bHMoBvKKKshEQJIhVxvQQQaNHEV0I5i02" X-Virus-Scanned: clamav-milter 0.99.2 at smtp.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 12:32:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --bHMoBvKKKshEQJIhVxvQQQaNHEV0I5i02 Content-Type: multipart/mixed; boundary="NSjAGd75kt2mS6xGBGgCoQLiA5FeVL52X" From: Matthew Seaman To: freebsd-net@freebsd.org Message-ID: Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> In-Reply-To: <2033B3FC769B74294656A089@[10.12.30.106]> --NSjAGd75kt2mS6xGBGgCoQLiA5FeVL52X Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 06/23/16 13:14, Karl Pielorz wrote: >=20 > --On 23 June 2016 11:53 +0100 Karl Pielorz wro= te: >=20 >> This gets increasingly weird if I run tcpdump on the 10.3 box. The act= of >> running 'tcpdump -i lagg1.30 -n' actually fixes the problem: >=20 > As a follow up - running 'ifconfig lagg1 promisc' fixes the issue as > well (as you'd kind of expect if tcpdump does while it's running). >=20 > I don't know if that's a good idea / workaround for now? I use a similar config with vlans over lagg. While I haven't seen exactly your problem, I did see one instance of the vlan interface coming up with an all-zero MAC address (out of about 10 systems upgraded to 10.3-RELEASE so far). See PR207701 -- my workaround was to explicitly set a MAC address for the vlan i/f. We're configuring the vlan interfaces slightly differently so that they end up with a name like 'vlan123' rather than 'lagg0.123' -- if that difference is significant then it maybe gives you an alternate workaround to running your interfaces promiscuously. Cheers, Matthew --NSjAGd75kt2mS6xGBGgCoQLiA5FeVL52X-- --bHMoBvKKKshEQJIhVxvQQQaNHEV0I5i02 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJXa9bQAAoJEABRPxDgqeTnVqsP+wWrcTctSxQ5xVMQsCTln664 tvHGgtQxnVGpN5WrxbrKCJQ49XaGoLx6v8HyKXQHE6My94huvW193M8VDYaYVVQx e51cr5i6+Z7wgkJVibBOi1/lOYxfz1lv9Nkt4+pe+laa31sntq7snVhlY7g8VNef LF98oKaYFiYrwzhmc75l68IqdMmzSfj03g3Pk967kza+6ChwmXgNfWNpWa5mJobV kJIl8xyQd9PqrHufaKvkD3jxHdP+wEcZnbQEMxXUYq6r1zd5neNn450Jg6jSGy9c 3x89E8XONtP2WXp9zM0VkZDU2E7NlT6Kjn9NWOnhQDEJ0V6vmxY9P4B0mTy/XmSz 2+dVefRY1K/GBXSsc9696ARmEbqrqyu+38GVLnMoHGFSz2iteZdxifqE0zUezMgg LJOwYR7TkP9SDG7xIGabFxWE27TxGOm7NJkvtQEstLg5NHITKgnNgJcVuZGot3d6 /wXNmQAoYZKH48zBYtP/qWzuo86DkTYF4VMkdF/ASL0BolwpXQTH4kgyoyxxYZZ6 xVtJBlY1Cb05Wtw3Ruh4h6AhGL7SYAX4advnIMQ6dXT8XKtOybZ1d6wV8hP3Az6q GNIyTZYvjmgbw++pjBaOAbqdbdG4esNuRBBG4u3t44fBzsXLxftHjgW1wkhyiil8 yVF0+EobYm3OCuepI1vI =3F4A -----END PGP SIGNATURE----- --bHMoBvKKKshEQJIhVxvQQQaNHEV0I5i02-- From owner-freebsd-net@freebsd.org Thu Jun 23 12:37:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08043A7A11C for ; Thu, 23 Jun 2016 12:37:05 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 5EE5C25C3 for ; Thu, 23 Jun 2016 12:37:03 +0000 (UTC) (envelope-from freebsd@grem.de) Received: (qmail 96891 invoked by uid 89); 23 Jun 2016 12:30:15 -0000 Received: from unknown (HELO ?192.168.101.124?) (mg@grem.de@194.97.158.66) by mail.grem.de with ESMTPA; 23 Jun 2016 12:30:15 -0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? From: Michael Gmelin X-Mailer: iPhone Mail (13F69) In-Reply-To: <2033B3FC769B74294656A089@[10.12.30.106]> Date: Thu, 23 Jun 2016 14:30:15 +0200 Cc: freebsd-net@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <5DA0293A-B07A-4975-A347-3DE91C5A6116@grem.de> References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> To: Karl Pielorz X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 12:37:05 -0000 > On 23 Jun 2016, at 14:14, Karl Pielorz wrote: >=20 >=20 > --On 23 June 2016 11:53 +0100 Karl Pielorz wrote:= >=20 >> This gets increasingly weird if I run tcpdump on the 10.3 box. The act of= >> running 'tcpdump -i lagg1.30 -n' actually fixes the problem: >=20 > As a follow up - running 'ifconfig lagg1 promisc' fixes the issue as well (= as you'd kind of expect if tcpdump does while it's running). >=20 > I don't know if that's a good idea / workaround for now? I don't think having an IP address on an interface that also has vlan interf= aces is a good idea. What kind of traffic are you expecting on lagg1? - Michael From owner-freebsd-net@freebsd.org Thu Jun 23 12:40:39 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2936DA7A1C5 for ; Thu, 23 Jun 2016 12:40:39 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7C85274D for ; Thu, 23 Jun 2016 12:40:38 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.15.2/8.15.2) with ESMTPSA id u5NCeZ2G063090 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Jun 2016 13:40:36 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Thu, 23 Jun 2016 13:40:27 +0100 From: Karl Pielorz To: Michael Gmelin cc: freebsd-net@FreeBSD.org Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: In-Reply-To: <5DA0293A-B07A-4975-A347-3DE91C5A6116@grem.de> References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> <5DA0293A-B07A-4975-A347-3DE91C5A6116@grem.de> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 12:40:39 -0000 --On 23 June 2016 14:30 +0200 Michael Gmelin wrote: > I don't think having an IP address on an interface that also has vlan > interfaces is a good idea. What kind of traffic are you expecting on > lagg1? Hi, This has worked for 'quite a while' (i.e. >year). lagg1 is connected to an HP switch that has on the two ports that are the lagg: - Untagged traffic (i.e. frames presented to the host with no VLAN tag, and frames expected back again with no VLAN tag). - Tagged VLAN30 - which is presented to the host tagged as VLAN30, and the switch expects to get frames back tagged with VLAN30 already. - Tagged VLAN35 - ditto but for VLAN35 The HP's don't seem to have an issue with it - and it worked before. The way rc.conf is interpreted at the moment means we end up without an IP on lagg1 anyway. If you do a manual 'ifconfig lagg1 inet 172.16.50.1 netmask 255.255.255.0' to fix this - it works (although the lagg1.30/35 VLAN's still don't work). In fact, as I just noticed before - if I do 'ifconfig lagg1 promisc' - it all works as it used to (on 10.1-R). -Kp From owner-freebsd-net@freebsd.org Thu Jun 23 13:04:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 034D9A7A883 for ; Thu, 23 Jun 2016 13:04:26 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A23B0174A; Thu, 23 Jun 2016 13:04:25 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.15.2/8.15.2) with ESMTPSA id u5ND4LQx064681 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Jun 2016 14:04:23 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Thu, 23 Jun 2016 14:04:13 +0100 From: Karl Pielorz To: Matthew Seaman , freebsd-net@freebsd.org Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: In-Reply-To: References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 13:04:26 -0000 --On 23 June 2016 13:32 +0100 Matthew Seaman wrote: > I use a similar config with vlans over lagg. While I haven't seen > exactly your problem, I did see one instance of the vlan interface > coming up with an all-zero MAC address (out of about 10 systems upgraded > to 10.3-RELEASE so far). See PR207701 -- my workaround was to > explicitly set a MAC address for the vlan i/f. Hi, We seem to always have a MAC address on the lagg - so I don't think we've hit that issue. > We're configuring the vlan interfaces slightly differently so that they > end up with a name like 'vlan123' rather than 'lagg0.123' -- if that > difference is significant then it maybe gives you an alternate > workaround to running your interfaces promiscuously. Any chance you can send us a snippet of how they're setup in '/etc/rc.conf' if it's different. When we originally set this up way-back-when we had a lot of fun with the various ways & syntax of getting it setup, just settling for the way we do 'because it works' - not ideal I guess, so I'd be interested to see if there's another way of spec'ing the config in rc.conf Cheers, -Karl From owner-freebsd-net@freebsd.org Thu Jun 23 13:15:06 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5A9DA7AAC7 for ; Thu, 23 Jun 2016 13:15:06 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 55F1C218C for ; Thu, 23 Jun 2016 13:15:05 +0000 (UTC) (envelope-from freebsd@grem.de) Received: (qmail 97380 invoked by uid 89); 23 Jun 2016 13:15:04 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@194.97.158.70) by mail.grem.de with ESMTPA; 23 Jun 2016 13:15:04 -0000 Date: Thu, 23 Jun 2016 15:15:03 +0200 From: Michael Gmelin To: Karl Pielorz Cc: Matthew Seaman , freebsd-net@freebsd.org Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: <20160623151503.1630324b@bsd64.grem.de> In-Reply-To: References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 13:15:07 -0000 On Thu, 23 Jun 2016 14:04:13 +0100 Karl Pielorz wrote: > --On 23 June 2016 13:32 +0100 Matthew Seaman > wrote: > > > I use a similar config with vlans over lagg. While I haven't seen > > exactly your problem, I did see one instance of the vlan interface > > coming up with an all-zero MAC address (out of about 10 systems > > upgraded to 10.3-RELEASE so far). See PR207701 -- my workaround > > was to explicitly set a MAC address for the vlan i/f. > > Hi, > > We seem to always have a MAC address on the lagg - so I don't think > we've hit that issue. > > > We're configuring the vlan interfaces slightly differently so that > > they end up with a name like 'vlan123' rather than 'lagg0.123' -- > > if that difference is significant then it maybe gives you an > > alternate workaround to running your interfaces promiscuously. > > Any chance you can send us a snippet of how they're setup in > '/etc/rc.conf' if it's different. > > When we originally set this up way-back-when we had a lot of fun with > the various ways & syntax of getting it setup, just settling for the > way we do 'because it works' - not ideal I guess, so I'd be > interested to see if there's another way of spec'ing the config in > rc.conf > Could you post the output of ifconfig after boot and while/after running tcpdump? -- Michael Gmelin From owner-freebsd-net@freebsd.org Thu Jun 23 13:41:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B515AC532E for ; Thu, 23 Jun 2016 13:41:14 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C49EE2E96; Thu, 23 Jun 2016 13:41:13 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.15.2/8.15.2) with ESMTPSA id u5NDfAj8067079 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Jun 2016 14:41:11 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Thu, 23 Jun 2016 14:41:02 +0100 From: Karl Pielorz To: Matthew Seaman , freebsd-net@freebsd.org cc: Michael Gmelin Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? Message-ID: <87C4B070AC631E5175831B3A@[10.12.30.106]> In-Reply-To: References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 13:41:14 -0000 Ok... --On 23 June 2016 13:32 +0100 Matthew Seaman wrote: > I use a similar config with vlans over lagg. While I haven't seen > exactly your problem, I did see one instance of the vlan interface > coming up with an all-zero MAC address (out of about 10 systems upgraded > to 10.3-RELEASE so far). See PR207701 -- my workaround was to > explicitly set a MAC address for the vlan i/f. Having just spent twenty minutes staring at MAC's on the machine - it *does appear* we are also seeing the same as PR207701. The machine has two lagg interfaces on it - I checked the wrong one initially (em based) - hence twenty minutes of head scratching and oh-so-long reboots (on HP kit) after I glimpsed one of them was indeed a MAC of 00:00:00:00:00:00 (which is on the bge based lagg vlans). I'll go read up on the PR, add myself to it, and implement the workaround. Sorry for the noise... Regards, -Karl From owner-freebsd-net@freebsd.org Thu Jun 23 14:17:43 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEC2FAC5C75 for ; Thu, 23 Jun 2016 14:17:43 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:c4ea:bd49:619b:6cb3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A54D21186 for ; Thu, 23 Jun 2016 14:17:43 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from ox-dell39.ox.adestra.com (unknown [85.199.232.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 8C94F2539; Thu, 23 Jun 2016 14:17:38 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/8C94F2539; dkim=none; dkim-atps=neutral Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? To: Karl Pielorz , freebsd-net@freebsd.org References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> From: Matthew Seaman Message-ID: Date: Thu, 23 Jun 2016 15:17:30 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="xxecjt33vip4I4xrAWW8Nc3BtoSCbcxOK" X-Virus-Scanned: clamav-milter 0.99.2 at smtp.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 14:17:44 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xxecjt33vip4I4xrAWW8Nc3BtoSCbcxOK Content-Type: multipart/mixed; boundary="co2nXkF1N5GKHxDA2AXB1kKmhmsLic8Ix" From: Matthew Seaman To: Karl Pielorz , freebsd-net@freebsd.org Message-ID: Subject: Re: Problem with VLAN config and traffic after 10.1-R -> 10.3-R-p5 Upgrade? References: <2ED5D9FEB55641BF734C14F3@[10.12.30.106]> <2033B3FC769B74294656A089@[10.12.30.106]> In-Reply-To: --co2nXkF1N5GKHxDA2AXB1kKmhmsLic8Ix Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 06/23/16 14:04, Karl Pielorz wrote: > Any chance you can send us a snippet of how they're setup in > '/etc/rc.conf' if it's different. It's all in the PR Cheers, Matthew --co2nXkF1N5GKHxDA2AXB1kKmhmsLic8Ix-- --xxecjt33vip4I4xrAWW8Nc3BtoSCbcxOK Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJXa+97AAoJEABRPxDgqeTngJIP/26szeuunBj8DSynxjKHeu2Q m9fqOzUW5fNYZrfMED+Vibud5qJr8wYGtyL4Oc4Yc0vgkJjnYn6wVDU5pOV5/eBr 86v98jLUd1aXhKCAVlF1qQfCJLNOuHM0s0zdl/Sh4Kq0Uwh8LDclo0zqXBGVRgGT hrbi73dhr6K2FjXbjtZ4V0G8nvbZc6onZyk5tPoEu94PlQlzXs9pGtj3hkD7mfJC 4JtP8oS7TRDQ3LZgcZ2gsImI66XF+yQicW/QOIdien0IktnTWneyi6hb1HUGWyQC wtuE3kwUsMDQCw9K4c9hsTzMmoIVYnl4eRh2hkj+vGLDXvg2Qyttj1YMGxBhEQYQ o5t6cmNa5VNzLcFEhReO6ygIwPVcKmf/R5PsBDFu7rPBdzcvqkpiG6QNWZ+9fdY6 otG4Y+i1QX+7yMcjcSKb1GNvkB4Nq7wkVLPIdVW/wOHMGf6KleFhAuKZgQraiPIj Pg68Awsn/IH0l0MP/PO8ifWQE8BTq7lFgNHfewlfl8KR3jNCMubj14tIFujeGZfr hVgEaVVoWIV2zhylmkW7kVJiIHTlBzrBEaGWzET7SgssvJYrWjBAlYKLFsWatCaq Os3tQhMHit1mQlaUDSx5WCsBvbJyI0WggfnDBvFexapsOiz3Ugpe7I++KnmREFyy ep5vrWo5G7aAU8F//1mO =UXRm -----END PGP SIGNATURE----- --xxecjt33vip4I4xrAWW8Nc3BtoSCbcxOK-- From owner-freebsd-net@freebsd.org Fri Jun 24 15:38:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 845A3B808B3 for ; Fri, 24 Jun 2016 15:38:03 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id E947D1D2C for ; Fri, 24 Jun 2016 15:38:02 +0000 (UTC) (envelope-from freebsd@grem.de) Received: (qmail 16275 invoked by uid 89); 24 Jun 2016 15:31:14 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@194.97.158.70) by mail.grem.de with ESMTPA; 24 Jun 2016 15:31:14 -0000 Date: Fri, 24 Jun 2016 17:31:13 +0200 From: Michael Gmelin To: freebsd-net@freebsd.org Subject: Re: ARP table entries / ifconfig needs to be issued twice when moving IP Message-ID: <20160624173113.2744f8bc@bsd64.grem.de> In-Reply-To: <20160621112538.784e7b6e@bsd64.grem.de> References: <20160621064800.GB8441@box-hlm-03.niklaas.eu> <20160621112538.784e7b6e@bsd64.grem.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jun 2016 15:38:03 -0000 An easier way to demonstrate the problem: When there is a temporary entry in the arp table, it's only overwritten by the permanent entry the second time ifconfig is called: 10.3-RELEASE / 10.2-RELEASE / 9.1-RELEASE: [root@ ~]# ifconfig bge0 10.1.1.1/24 [root@ ~]# arp -s 10.1.1.2 14:18:77:00:00:00 temp [root@ ~]# ifconfig bge0 alias 10.1.1.2/32 [root@ ~]# arp 10.1.1.2 ? (10.1.1.2) at 14:18:77:00:00:00 on bge0 expires in 1178 seconds [root@ ~]# # ifconfig bge0 alias 10.1.1.2/32 ? (10.1.1.2) at 14:18:77:4d:10:61 on bge0 permanent [ethernet] - Michael -- Michael Gmelin From owner-freebsd-net@freebsd.org Sat Jun 25 08:48:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFB90B7324A for ; Sat, 25 Jun 2016 08:48:55 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD2C1D6C for ; Sat, 25 Jun 2016 08:48:55 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mailman.ysv.freebsd.org (Postfix) id 89381B73248; Sat, 25 Jun 2016 08:48:55 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88E64B73247 for ; Sat, 25 Jun 2016 08:48:55 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 412E61D66 for ; Sat, 25 Jun 2016 08:48:55 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mail-vk0-x22c.google.com with SMTP id c2so148289086vkg.1 for ; Sat, 25 Jun 2016 01:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=tpBM99m1BKG3QPoXTcRmsr7jEgv/ta+6xDZKRAKia1Y=; b=bCEpaLnVowICR96R1x9BJjRu7d6E5Byl+h3uxJYmaq7ApBRu3iQkeTJQ6ZtxQq1isW 6Fyj4Pa8fBkRdeE8goX1wGSm3N0kT+LPitT9RkwPGiqq0PtsbH417Hng2JTFmlMT2qQ/ 49OWYk7EJXyEVgvpsHz63CIfaPGy4lRC8tR2A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=tpBM99m1BKG3QPoXTcRmsr7jEgv/ta+6xDZKRAKia1Y=; b=PxwGlONpRu/CI21sZgTzPqgbiwY71zBzbrpSMSX7lgopUm7OqNLcGu1R9gk2KfOG5w xBqDWNfPU6MMrKj6KCMhIIIAcJL0tLDUiFatCWThUGE22BE31JfV5rqyRwnJ/e5jia2F M9I3bDTR8VfunhVSTLCT9AvT2e0N2n898tLf9OuBsrzsXH5gBgdbsFxMp0raEyF5sfun Q6op541GzDo7ZrBHRadERtKU6sbnjS25Z8bHhB+czzAaGir+kFBxf9QLy4BXS8HYrlDa h+tb7wbbzkq8EgAC7Y8YwbrrPtp+ml7ZfSuQk8R7B2wpar650n1DTa4I/mg55E9JKldG ie6Q== X-Gm-Message-State: ALyK8tIvqzZbrYJ8ykPMSUs7Oep0rrSkMSt8b0e1XCDTtJiKHocCQkUNffKULtWQtufDhvb1 X-Received: by 10.176.64.7 with SMTP id h7mr4365555uad.153.1466844534204; Sat, 25 Jun 2016 01:48:54 -0700 (PDT) Received: from [100.127.86.242] ([69.53.246.16]) by smtp.gmail.com with ESMTPSA id g4sm2820025vkb.3.2016.06.25.01.48.52 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 25 Jun 2016 01:48:53 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: panic with tcp timers From: Randall Stewart In-Reply-To: <74bb31b7-a9f5-3d0c-eea0-681872e6f09b@freebsd.org> Date: Sat, 25 Jun 2016 04:48:51 -0400 Cc: Gleb Smirnoff , Randall Ray Stewart , hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <18D94615-810E-4E79-A889-4B0CC70F9E45@netflix.com> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <74bb31b7-a9f5-3d0c-eea0-681872e6f09b@freebsd.org> To: Julien Charbon X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 08:48:55 -0000 So All of our timers in TCP do something like --------------------- INFO-LOCK INP_WLOCK if (inp needs to be dropped) { drop-it } do other work UNLOCK-INP UNLOCK-INFO ------------------ And generally the path =93inp needs to be dropped=94 is rarely taken. So why don=92t we change the procedure to something like: INP_WLOCK if (inp needs to be dropped) { inp_ref() INP_WUNLOCK() INFO_LOCK() INP_WLOCK() if (inp_release_ref) { /* victory its dead */ INFO_UNLOCK return } do-the-drop } This way we would only go grab the INFO lock in those rarer cases when we *do* actually want to kill the tcb and at the same time it would make the current callout system work correctly.. which as many have pointed out would be much better if we could just let the lock be gotten by the callout system. Hmm maybe with this scheme we could even do that... R > On Jun 20, 2016, at 1:45 PM, Julien Charbon wrote: >=20 >=20 > Hi, >=20 > On 6/20/16 11:58 AM, Gleb Smirnoff wrote: >> On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote: >> J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: >> J> > J> > Comparing stable/10 and head, I see two changes that could >> J> > J> > affect that: >> J> > J> >=20 >> J> > J> > - callout_async_drain >> J> > J> > - switch to READ lock for inp info in tcp timers >> J> > J> >=20 >> J> > J> > That's why you are in To, Julien and Hans :) >> J> > J> >=20 >> J> > J> > We continue investigating, and I will keep you updated. >> J> > J> > However, any help is welcome. I can share cores. >> J> >=20 >> J> > Now, spending some time with cores and adding a bunch of >> J> > extra CTRs, I have a sequence of events that lead to the >> J> > panic. In short, the bug is in the callout system. It seems >> J> > to be not relevant to the callout_async_drain, at least for >> J> > now. The transition to READ lock unmasked the problem, that's >> J> > why NetflixBSD 10 doesn't panic. >> J> >=20 >> J> > The panic requires heavy contention on the TCP info lock. >> J> >=20 >> J> > [CPU 1] the callout fires, tcp_timer_keep entered >> J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); >> J> > [CPU 2] schedules the callout >> J> > [CPU 2] tcp_discardcb called >> J> > [CPU 2] callout successfully canceled >> J> > [CPU 2] tcpcb freed >> J> > [CPU 1] unblocks... panic >> J> >=20 >> J> > When the lock was WLOCK, all contenders were resumed in a >> J> > sequence they came to the lock. Now, that they are readers, >> J> > once the lock is released, readers are resumed in a "random" >> J> > order, and this allows tcp_discardcb to go before the old >> J> > running callout, and this unmasks the panic. >> J>=20 >> J> Highly interesting. I should be able to reproduce that (will be = useful >> J> for testing the corresponding fix). >> J>=20 >> J> Fix proposal: If callout_async_drain() returns 0 (fail) (instead = of 1 >> J> (success) here) when the callout cancellation is a success _but_ = the >> J> callout is current running, that should fix it. >> J>=20 >> J> For the history: It comes back to my old callout question: >> J>=20 >> J> Does _callout_stop_safe() is allowed to return 1 (success) even = if the >> J> callout is still currently running; a.k.a. it is not because you >> J> successfully cancelled a callout that the callout is not currently = running. >> J>=20 >> J> We did propose a patch to make _callout_stop_safe() returns 0 = (fail) >> J> when the callout is currently running: >> J>=20 >> J> callout_stop() should return 0 when the callout is currently being >> J> serviced and indeed unstoppable >> J> = https://reviews.freebsd.org/differential/changeset/?ref=3D62513&whitespace= =3Dignore-most >> J>=20 >> J> But this change impacted too many old code paths and was = interesting >> J> only for TCP timers and thus was abandoned. >>=20 >> The fix I am working on now is doing exactly that. callout_reset must >> return 0 if the callout is currently running. >>=20 >> What are the old paths impacted? >=20 > Actually all the paths that check the callout_stop() return value AND > call both callout_reset() and callout_stop() AND use mpsafe callout(). > And for each path, we would have to check our patch was ok (or not). >=20 > Thus, if you only do the change in callout_async_drain() context, you > don't impact the "old" callout_stop() behavior and get the desired > behavior for the TCP timers. Might be a good trade-off... >=20 > My 2 cents. >=20 > -- > Julien -------- Randall Stewart rrs@netflix.com 803-317-4952 From owner-freebsd-net@freebsd.org Sat Jun 25 14:41:27 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E85F6B80463 for ; Sat, 25 Jun 2016 14:41:27 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BE3871B44 for ; Sat, 25 Jun 2016 14:41:27 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mailman.ysv.freebsd.org (Postfix) id BD410B8045E; Sat, 25 Jun 2016 14:41:27 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCD2EB8045D for ; Sat, 25 Jun 2016 14:41:27 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mail-vk0-x234.google.com (mail-vk0-x234.google.com [IPv6:2607:f8b0:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6891A1B3D for ; Sat, 25 Jun 2016 14:41:27 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mail-vk0-x234.google.com with SMTP id u64so184605727vkf.3 for ; Sat, 25 Jun 2016 07:41:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=from:mime-version:subject:in-reply-to:date:cc:message-id:references :to; bh=Duy9BnwOmcR9mCL+dw5z6on0vanDV3plsSQCxKfAZjQ=; b=DDgWUVly6boLZ4Y6k9C1+lCp+YQDoUSLcfYK/iupnpZClD2e4hJICYOigCL2WUkczS 3li1Q2JBYtjgNbyRFC3e5HCwtrFz/+znhMu1LbSbxIZvP3nak4MV73X5EG30Yhc2wGrt jP/QIUf1f7kDTFcrPXd8Vyoxw7El9LgsG3bZE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:mime-version:subject:in-reply-to:date:cc :message-id:references:to; bh=Duy9BnwOmcR9mCL+dw5z6on0vanDV3plsSQCxKfAZjQ=; b=A7kx2rx1PyRhafsesXY4auUtfkhzWtvWgiiIWkru/4viLE5ghwb8PTwzw3dbxIciXA w570XDa3IIDMwHMjz/i8Cs7HzsBejHNA36y13WXugUiwbxIt1QY0BL4r7ntGr5WG/0k6 XLKSUiSP7s1A8u6vJyO37/Px+pzgWz3bwnyU3FnLXaGbf85khC06G6nd4BPTucKds1k1 zUGlnFq2E3kNc0ayPRC5mao9SdC3guaxMQFI9n3CcPprYouVjXIaGJaAt50jKc4hsY1V cY12Z2ZFjOFWsfzYot3qJL5n3gQgRboeLX9w5pfIT2HJyXWl3aXcLdkHydL0vMiyw4d1 5ikw== X-Gm-Message-State: ALyK8tLxXSFdd/x1sb6lLbB/01qZAGCoCoTzyrtjdoBtROnu4+8w8Usyn6eNKY2YBw1W/rU1 X-Received: by 10.176.4.49 with SMTP id 46mr4269844uav.32.1466865686298; Sat, 25 Jun 2016 07:41:26 -0700 (PDT) Received: from [100.127.86.242] ([69.53.246.16]) by smtp.gmail.com with ESMTPSA id 45sm2914659uaq.6.2016.06.25.07.41.24 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 25 Jun 2016 07:41:25 -0700 (PDT) From: Randall Stewart X-Google-Original-From: Randall Stewart Content-Type: multipart/mixed; boundary="Apple-Mail=_F9B158CC-A33B-400F-9DD1-C3982E64FC31" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: panic with tcp timers In-Reply-To: <18D94615-810E-4E79-A889-4B0CC70F9E45@netflix.com> Date: Sat, 25 Jun 2016 10:41:24 -0400 Cc: Julien Charbon , Gleb Smirnoff , hselasky@FreeBSD.org, net@FreeBSD.org Message-Id: <6E52CA6A-2153-4EF9-A3E1-97CB0D07EB28@freebsd.org> References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> <20160620095822.GJ1076@FreeBSD.org> <74bb31b7-a9f5-3d0c-eea0-681872e6f09b@freebsd.org> <18D94615-810E-4E79-A889-4B0CC70F9E45@netflix.com> To: current@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 14:41:28 -0000 --Apple-Mail=_F9B158CC-A33B-400F-9DD1-C3982E64FC31 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Ok Lets try this again with my source changed to my @freebsd.net :-) Now I am also attaching a patch for you Gleb, this will take some poking = to get in to your NF-head since it incorporates some changes we made = earlier. I think this will fix the problem.. i.e. dealing with two locks in the = callout system (which it was never meant to have done).. Note we probably can move the code to use the callout lock init now.. = but lets see if this works on your setup on c096 and if so we can think about doing that. R --Apple-Mail=_F9B158CC-A33B-400F-9DD1-C3982E64FC31 Content-Disposition: attachment; filename=for_gleb Content-Type: application/octet-stream; name="for_gleb" Content-Transfer-Encoding: 7bit Index: tcp_timer.c =================================================================== --- tcp_timer.c (revision 302198) +++ tcp_timer.c (working copy) @@ -306,6 +306,33 @@ tcp_timer_delack(void *xtp) CURVNET_RESTORE(); } +static inline int +tcp_inp_lock_exchange(struct tcpcb *tp) +{ + struct inpcb *inp; + + + inp = tp->t_inpcb; + in_pcbref(inp); + INP_WUNLOCK(inp); + INP_INFO_RLOCK(&V_tcbinfo); + INP_WLOCK(inp); + if (inp->inp_flags & (INP_TIMEWAIT | INP_DROPPED)) { + return(1); + } + return(0); + +} + +static inline void +tcp_inp_unlock_exchange(struct tcpcb *tp) +{ + INP_INFO_RUNLOCK(&V_tcbinfo); + if (tp && in_pcbrele_wlocked(tp->t_inpcb) == 0) { + INP_WUNLOCK(tp->t_inpcb); + } +} + void tcp_timer_2msl(void *xtp) { @@ -317,7 +344,6 @@ tcp_timer_2msl(void *xtp) ostate = tp->t_state; #endif - INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); INP_WLOCK(inp); @@ -325,7 +351,6 @@ tcp_timer_2msl(void *xtp) if (callout_pending(&tp->t_timers->tt_2msl) || !callout_active(&tp->t_timers->tt_2msl)) { INP_WUNLOCK(tp->t_inpcb); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -332,7 +357,6 @@ tcp_timer_2msl(void *xtp) callout_deactivate(&tp->t_timers->tt_2msl); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -355,7 +379,6 @@ tcp_timer_2msl(void *xtp) */ if ((inp->inp_flags & INP_TIMEWAIT) != 0) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -363,7 +386,13 @@ tcp_timer_2msl(void *xtp) tp->t_inpcb && tp->t_inpcb->inp_socket && (tp->t_inpcb->inp_socket->so_rcv.sb_state & SBS_CANTRCVMORE)) { TCPSTAT_INC(tcps_finwait2_drops); + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } tp = tcp_close(tp); + tcp_inp_unlock_exchange(tp); + goto out; } else { if (ticks - tp->t_rcvtime <= TP_MAXIDLE(tp)) { if (!callout_reset(&tp->t_timers->tt_2msl, @@ -370,8 +399,15 @@ tcp_timer_2msl(void *xtp) TP_KEEPINTVL(tp), tcp_timer_2msl, tp)) { tp->t_timers->tt_flags &= ~TT_2MSL_RST; } - } else - tp = tcp_close(tp); + } else { + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } + tp = tcp_close(tp); + tcp_inp_unlock_exchange(tp); + goto out; + } } #ifdef TCPDEBUG @@ -383,7 +419,7 @@ tcp_timer_2msl(void *xtp) if (tp != NULL) INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); +out: CURVNET_RESTORE(); } @@ -399,7 +435,6 @@ tcp_timer_keep(void *xtp) ostate = tp->t_state; #endif - INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); INP_WLOCK(inp); @@ -406,7 +441,6 @@ tcp_timer_keep(void *xtp) if (callout_pending(&tp->t_timers->tt_keep) || !callout_active(&tp->t_timers->tt_keep)) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -413,7 +447,6 @@ tcp_timer_keep(void *xtp) callout_deactivate(&tp->t_timers->tt_keep); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -468,12 +501,16 @@ tcp_timer_keep(void *xtp) #endif TCP_PROBE2(debug__user, tp, PRU_SLOWTIMO); INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; dropit: TCPSTAT_INC(tcps_keepdrops); + + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } tp = tcp_drop(tp, ETIMEDOUT); #ifdef TCPDEBUG @@ -482,9 +519,8 @@ dropit: PRU_SLOWTIMO); #endif TCP_PROBE2(debug__user, tp, PRU_SLOWTIMO); - if (tp != NULL) - INP_WUNLOCK(tp->t_inpcb); - INP_INFO_RUNLOCK(&V_tcbinfo); + tcp_inp_unlock_exchange(tp); +out: CURVNET_RESTORE(); } @@ -499,7 +535,6 @@ tcp_timer_persist(void *xtp) ostate = tp->t_state; #endif - INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); INP_WLOCK(inp); @@ -506,7 +541,6 @@ tcp_timer_persist(void *xtp) if (callout_pending(&tp->t_timers->tt_persist) || !callout_active(&tp->t_timers->tt_persist)) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -513,7 +547,6 @@ tcp_timer_persist(void *xtp) callout_deactivate(&tp->t_timers->tt_persist); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -537,7 +570,12 @@ tcp_timer_persist(void *xtp) (ticks - tp->t_rcvtime >= tcp_maxpersistidle || ticks - tp->t_rcvtime >= TCP_REXMTVAL(tp) * tcp_totbackoff)) { TCPSTAT_INC(tcps_persistdrop); + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } tp = tcp_drop(tp, ETIMEDOUT); + tcp_inp_unlock_exchange(tp); goto out; } /* @@ -547,7 +585,12 @@ tcp_timer_persist(void *xtp) if (tp->t_state > TCPS_CLOSE_WAIT && (ticks - tp->t_rcvtime) >= TCPTV_PERSMAX) { TCPSTAT_INC(tcps_persistdrop); + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } tp = tcp_drop(tp, ETIMEDOUT); + tcp_inp_unlock_exchange(tp); goto out; } tcp_setpersist(tp); @@ -555,15 +598,13 @@ tcp_timer_persist(void *xtp) (void) tp->t_fb->tfb_tcp_output(tp); tp->t_flags &= ~TF_FORCEDATA; -out: #ifdef TCPDEBUG if (tp != NULL && tp->t_inpcb->inp_socket->so_options & SO_DEBUG) tcp_trace(TA_USER, ostate, tp, NULL, NULL, PRU_SLOWTIMO); #endif TCP_PROBE2(debug__user, tp, PRU_SLOWTIMO); - if (tp != NULL) - INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_WUNLOCK(inp); +out: CURVNET_RESTORE(); } @@ -573,7 +614,6 @@ tcp_timer_rexmt(void * xtp) struct tcpcb *tp = xtp; CURVNET_SET(tp->t_vnet); int rexmt; - int headlocked; struct inpcb *inp; #ifdef TCPDEBUG int ostate; @@ -580,8 +620,6 @@ tcp_timer_rexmt(void * xtp) ostate = tp->t_state; #endif - - INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); INP_WLOCK(inp); @@ -588,7 +626,6 @@ tcp_timer_rexmt(void * xtp) if (callout_pending(&tp->t_timers->tt_rexmt) || !callout_active(&tp->t_timers->tt_rexmt)) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -595,7 +632,6 @@ tcp_timer_rexmt(void * xtp) callout_deactivate(&tp->t_timers->tt_rexmt); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -616,14 +652,15 @@ tcp_timer_rexmt(void * xtp) if (++tp->t_rxtshift > TCP_MAXRXTSHIFT) { tp->t_rxtshift = TCP_MAXRXTSHIFT; TCPSTAT_INC(tcps_timeoutdrop); - + if (tcp_inp_lock_exchange(tp)) { + tcp_inp_unlock_exchange(tp); + goto out; + } tp = tcp_drop(tp, tp->t_softerror ? tp->t_softerror : ETIMEDOUT); - headlocked = 1; + tcp_inp_unlock_exchange(tp); goto out; } - INP_INFO_RUNLOCK(&V_tcbinfo); - headlocked = 0; if (tp->t_state == TCPS_SYN_SENT) { /* * If the SYN was retransmitted, indicate CWND to be @@ -811,7 +848,6 @@ tcp_timer_rexmt(void * xtp) (void) tp->t_fb->tfb_tcp_output(tp); -out: #ifdef TCPDEBUG if (tp != NULL && (tp->t_inpcb->inp_socket->so_options & SO_DEBUG)) tcp_trace(TA_USER, ostate, tp, (void *)0, (struct tcphdr *)0, @@ -818,10 +854,8 @@ tcp_timer_rexmt(void * xtp) PRU_SLOWTIMO); #endif TCP_PROBE2(debug__user, tp, PRU_SLOWTIMO); - if (tp != NULL) - INP_WUNLOCK(inp); - if (headlocked) - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_WUNLOCK(inp); +out: CURVNET_RESTORE(); } @@ -832,7 +866,6 @@ tcp_timer_activate(struct tcpcb *tp, uint32_t time timeout_t *f_callout; struct inpcb *inp = tp->t_inpcb; int cpu = inp_to_cpuid(inp); - uint32_t f_reset; #ifdef TCP_OFFLOAD if (tp->t_flags & TF_TOE) @@ -846,27 +879,22 @@ tcp_timer_activate(struct tcpcb *tp, uint32_t time case TT_DELACK: t_callout = &tp->t_timers->tt_delack; f_callout = tcp_timer_delack; - f_reset = TT_DELACK_RST; break; case TT_REXMT: t_callout = &tp->t_timers->tt_rexmt; f_callout = tcp_timer_rexmt; - f_reset = TT_REXMT_RST; break; case TT_PERSIST: t_callout = &tp->t_timers->tt_persist; f_callout = tcp_timer_persist; - f_reset = TT_PERSIST_RST; break; case TT_KEEP: t_callout = &tp->t_timers->tt_keep; f_callout = tcp_timer_keep; - f_reset = TT_KEEP_RST; break; case TT_2MSL: t_callout = &tp->t_timers->tt_2msl; f_callout = tcp_timer_2msl; - f_reset = TT_2MSL_RST; break; default: if (tp->t_fb->tfb_tcp_timer_activate) { @@ -876,24 +904,9 @@ tcp_timer_activate(struct tcpcb *tp, uint32_t time panic("tp %p bad timer_type %#x", tp, timer_type); } if (delta == 0) { - if ((tp->t_timers->tt_flags & timer_type) && - (callout_stop(t_callout) > 0) && - (tp->t_timers->tt_flags & f_reset)) { - tp->t_timers->tt_flags &= ~(timer_type | f_reset); - } + callout_stop(t_callout); } else { - if ((tp->t_timers->tt_flags & timer_type) == 0) { - tp->t_timers->tt_flags |= (timer_type | f_reset); - callout_reset_on(t_callout, delta, f_callout, tp, cpu); - } else { - /* Reset already running callout on the same CPU. */ - if (!callout_reset(t_callout, delta, f_callout, tp)) { - /* - * Callout not cancelled, consider it as not - * properly restarted. */ - tp->t_timers->tt_flags &= ~f_reset; - } - } + callout_reset_on(t_callout, delta, f_callout, tp, cpu); } } @@ -931,30 +944,23 @@ void tcp_timer_stop(struct tcpcb *tp, uint32_t timer_type) { struct callout *t_callout; - uint32_t f_reset; tp->t_timers->tt_flags |= TT_STOPPED; - switch (timer_type) { case TT_DELACK: t_callout = &tp->t_timers->tt_delack; - f_reset = TT_DELACK_RST; break; case TT_REXMT: t_callout = &tp->t_timers->tt_rexmt; - f_reset = TT_REXMT_RST; break; case TT_PERSIST: t_callout = &tp->t_timers->tt_persist; - f_reset = TT_PERSIST_RST; break; case TT_KEEP: t_callout = &tp->t_timers->tt_keep; - f_reset = TT_KEEP_RST; break; case TT_2MSL: t_callout = &tp->t_timers->tt_2msl; - f_reset = TT_2MSL_RST; break; default: if (tp->t_fb->tfb_tcp_timer_stop) { @@ -968,15 +974,13 @@ tcp_timer_stop(struct tcpcb *tp, uint32_t timer_ty panic("tp %p bad timer_type %#x", tp, timer_type); } - if (tp->t_timers->tt_flags & timer_type) { - if (callout_async_drain(t_callout, tcp_timer_discard) == 0) { - /* - * Can't stop the callout, defer tcpcb actual deletion - * to the last one. We do this using the async drain - * function and incrementing the count in - */ - tp->t_timers->tt_draincnt++; - } + if (callout_async_drain(t_callout, tcp_timer_discard) == 0) { + /* + * Can't stop the callout, defer tcpcb actual deletion + * to the last one. We do this using the async drain + * function and incrementing the count in + */ + tp->t_timers->tt_draincnt++; } } --Apple-Mail=_F9B158CC-A33B-400F-9DD1-C3982E64FC31 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > On Jun 25, 2016, at 4:48 AM, Randall Stewart wrote: >=20 > So >=20 > All of our timers in TCP do something like > --------------------- > INFO-LOCK > INP_WLOCK >=20 > if (inp needs to be dropped) { > drop-it > } > do other work >=20 > UNLOCK-INP > UNLOCK-INFO > ------------------ >=20 > And generally the path =93inp needs to be dropped=94 is rarely taken. >=20 > So why don=92t we change the procedure to something like: >=20 > INP_WLOCK > if (inp needs to be dropped) { > inp_ref() > INP_WUNLOCK() > INFO_LOCK() > INP_WLOCK() > if (inp_release_ref) { > /* victory its dead */ > INFO_UNLOCK > return > } > do-the-drop > } >=20 > This way we would only go grab the INFO lock in those rarer cases > when we *do* actually want to kill the tcb and at the same time > it would make the current callout system work correctly.. which as > many have pointed out would be much better if we could just let the > lock be gotten by the callout system. Hmm maybe with this scheme we > could even do that... >=20 > R >=20 >=20 >> On Jun 20, 2016, at 1:45 PM, Julien Charbon wrote: >>=20 >>=20 >> Hi, >>=20 >> On 6/20/16 11:58 AM, Gleb Smirnoff wrote: >>> On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote: >>> J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: >>> J> > J> > Comparing stable/10 and head, I see two changes that could >>> J> > J> > affect that: >>> J> > J> >=20 >>> J> > J> > - callout_async_drain >>> J> > J> > - switch to READ lock for inp info in tcp timers >>> J> > J> >=20 >>> J> > J> > That's why you are in To, Julien and Hans :) >>> J> > J> >=20 >>> J> > J> > We continue investigating, and I will keep you updated. >>> J> > J> > However, any help is welcome. I can share cores. >>> J> >=20 >>> J> > Now, spending some time with cores and adding a bunch of >>> J> > extra CTRs, I have a sequence of events that lead to the >>> J> > panic. In short, the bug is in the callout system. It seems >>> J> > to be not relevant to the callout_async_drain, at least for >>> J> > now. The transition to READ lock unmasked the problem, that's >>> J> > why NetflixBSD 10 doesn't panic. >>> J> >=20 >>> J> > The panic requires heavy contention on the TCP info lock. >>> J> >=20 >>> J> > [CPU 1] the callout fires, tcp_timer_keep entered >>> J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); >>> J> > [CPU 2] schedules the callout >>> J> > [CPU 2] tcp_discardcb called >>> J> > [CPU 2] callout successfully canceled >>> J> > [CPU 2] tcpcb freed >>> J> > [CPU 1] unblocks... panic >>> J> >=20 >>> J> > When the lock was WLOCK, all contenders were resumed in a >>> J> > sequence they came to the lock. Now, that they are readers, >>> J> > once the lock is released, readers are resumed in a "random" >>> J> > order, and this allows tcp_discardcb to go before the old >>> J> > running callout, and this unmasks the panic. >>> J>=20 >>> J> Highly interesting. I should be able to reproduce that (will be = useful >>> J> for testing the corresponding fix). >>> J>=20 >>> J> Fix proposal: If callout_async_drain() returns 0 (fail) = (instead of 1 >>> J> (success) here) when the callout cancellation is a success _but_ = the >>> J> callout is current running, that should fix it. >>> J>=20 >>> J> For the history: It comes back to my old callout question: >>> J>=20 >>> J> Does _callout_stop_safe() is allowed to return 1 (success) even = if the >>> J> callout is still currently running; a.k.a. it is not because you >>> J> successfully cancelled a callout that the callout is not = currently running. >>> J>=20 >>> J> We did propose a patch to make _callout_stop_safe() returns 0 = (fail) >>> J> when the callout is currently running: >>> J>=20 >>> J> callout_stop() should return 0 when the callout is currently = being >>> J> serviced and indeed unstoppable >>> J> = https://reviews.freebsd.org/differential/changeset/?ref=3D62513&whitespace= =3Dignore-most >>> J>=20 >>> J> But this change impacted too many old code paths and was = interesting >>> J> only for TCP timers and thus was abandoned. >>>=20 >>> The fix I am working on now is doing exactly that. callout_reset = must >>> return 0 if the callout is currently running. >>>=20 >>> What are the old paths impacted? >>=20 >> Actually all the paths that check the callout_stop() return value AND >> call both callout_reset() and callout_stop() AND use mpsafe = callout(). >> And for each path, we would have to check our patch was ok (or not). >>=20 >> Thus, if you only do the change in callout_async_drain() context, you >> don't impact the "old" callout_stop() behavior and get the desired >> behavior for the TCP timers. Might be a good trade-off... >>=20 >> My 2 cents. >>=20 >> -- >> Julien >=20 > -------- > Randall Stewart > rrs@netflix.com > 803-317-4952 >=20 >=20 >=20 >=20 >=20 -------- Randall Stewart rrs@netflix.com 803-317-4952 --Apple-Mail=_F9B158CC-A33B-400F-9DD1-C3982E64FC31-- From owner-freebsd-net@freebsd.org Sat Jun 25 18:38:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B718B81F7E for ; Sat, 25 Jun 2016 18:38:00 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) Received: from jackal.cherry.relay.mailchannels.net (jackal.cherry.relay.mailchannels.net [23.83.223.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46B722C37 for ; Sat, 25 Jun 2016 18:37:56 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id AFEA5A0C35 for ; Sat, 25 Jun 2016 16:42:43 +0000 (UTC) Received: from bs3-dallas.accountservergroup.com (ip-10-213-0-221.us-west-2.compute.internal [10.213.0.221]) by relay.mailchannels.net (Postfix) with ESMTPA id EC68BA0C9C for ; Sat, 25 Jun 2016 16:42:42 +0000 (UTC) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from bs3-dallas.accountservergroup.com (bs3-dallas.accountservergroup.com [10.92.147.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.6.15); Sat, 25 Jun 2016 16:42:43 +0000 X-MC-Relay: Forwarding X-MailChannels-SenderId: _forwarded-from|212.69.61.187 X-MailChannels-Auth-Id: wwwh X-MC-Loop-Signature: 1466872963156:1480989450 X-MC-Ingress-Time: 1466872963156 Received: from cust187-dsl61.idnet.net ([212.69.61.187]:51868 helo=copperhead.int.arc7.info) by bs3-dallas.accountservergroup.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.87) (envelope-from ) id 1bGqfJ-0000VA-Ry for freebsd-net@freebsd.org; Sat, 25 Jun 2016 11:42:42 -0500 Date: Sat, 25 Jun 2016 16:42:40 +0000 From: To: freebsd-net@freebsd.org Subject: ifconfig: BRDGADD lo1: invalid argument Message-ID: <20160625164240.7cea7587@copperhead.int.arc7.info> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PopBeforeSMTPSenders: org.freebsd.security@io7m.com X-AuthUser: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 18:38:00 -0000 Hello. I'm trying to create a bridge interface to isolate some jails on private addresses. I'm on a near-pristine install of 10.3, updated to 10.3-p5 via freebsd-update. The virtual interface to which the jails will be bound: # ifconfig lo1 create The bridge: # ifconfig bridge create bridge0 # ifconfig bridge0 addm em0 # ifconfig bridge0 addm lo1 ifconfig: BRDGADD lo1: Invalid argument I can find one reference to this error message online from 2007: https://lists.freebsd.org/pipermail/freebsd-net/2007-December/016102.html ... but this doesn't appear to be the problem I'm having (it seems as though it was caused by an out-of-sync kernel and world, which I obviously don't have). Any ideas how to get this working? M From owner-freebsd-net@freebsd.org Sat Jun 25 19:16:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E151AB806DC for ; Sat, 25 Jun 2016 19:16:18 +0000 (UTC) (envelope-from marieheleneka@gmail.com) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 794BA116E for ; Sat, 25 Jun 2016 19:16:18 +0000 (UTC) (envelope-from marieheleneka@gmail.com) Received: by mail-wm0-x22e.google.com with SMTP id r201so64211537wme.1 for ; Sat, 25 Jun 2016 12:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=TXiYw8q70yaoq8fsO10rJDRKKHj6lwAPxJdLx96OXzQ=; b=mzd0SRlj9LZCIff10EJgBdngn3W6dDAr5gK4W+DD0SubD4Amf+H58IaeoKmTynlruN xJe0lMYurQXTc1y40zHT0dIS6TgSgWqLJfM4Y8d7szGqvjzJW46s7cG1hyi6cNQC25Kl dDLkbBu22zi9k9nFubcCUUpMxajUVKTR36srkKaikfDMSOf7ws1oi+gvbqiVCXd0/4bf pm6btdDNhtv0NGK72HAD3QzyiDFN7vBjrgVUKo223SMcFf1+iXP7hAqA11NW6q/cSdzF 6jLEhMMI7wzSJCFGwKd3Xbm+eobs3n9SODTDaH3V1mElEtnGMnah1uBSdkni6qFUmgez dJEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=TXiYw8q70yaoq8fsO10rJDRKKHj6lwAPxJdLx96OXzQ=; b=DBnKC+W87AWlm+8R26HkC91sClhijsvJ/LuOiIyRHV0F2C9k7V3MR751yXbRMzp9BK pheErnrjzZkeY9GajeLdEycHUuu8T0KhzZoEHxVup2z5CCVk3mNGuxug2nXORw+aSLKk FZHj5ZIU1l4Bak2ULg7FWuyXoCk5ox+H+vLYw9cR2qWyAFDbkF0CZ5nzp9NgEC5nM5E2 aSId5HUtmN6WgyeCDah9IChtYp6+0rsVeaeKSm4lom9BlpmKJ2XAIRkmqBhfN+LHmQOZ fEaEsYUGlJGOzFRQAvWe5GasOVsSKaCyJD8UW4yr/qUyaaXPYVngvfhBopaPjtWylgrM FVIA== X-Gm-Message-State: ALyK8tLFQGAzsx2xkefxccwkFKLWpULKEgnq0ZJrGVpnwR08En4Bv9tUj6fv+1p12M3f9TsAyYNc0Wu4kjcI1w== X-Received: by 10.194.157.162 with SMTP id wn2mr10001705wjb.103.1466882176407; Sat, 25 Jun 2016 12:16:16 -0700 (PDT) MIME-Version: 1.0 References: <20160625164240.7cea7587@copperhead.int.arc7.info> In-Reply-To: <20160625164240.7cea7587@copperhead.int.arc7.info> From: Marie Helene Kvello-Aune Date: Sat, 25 Jun 2016 19:16:06 +0000 Message-ID: Subject: Re: ifconfig: BRDGADD lo1: invalid argument To: org.freebsd.security@io7m.com, freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 19:16:19 -0000 Check that lo1 has same MTU as bridge0. Regards, Marie Helene On Sat, Jun 25, 2016 at 8:38 PM wrote: > Hello. > > I'm trying to create a bridge interface to isolate some jails on > private addresses. I'm on a near-pristine install of 10.3, updated to > 10.3-p5 via freebsd-update. > > The virtual interface to which the jails will be bound: > > # ifconfig lo1 create > > The bridge: > > # ifconfig bridge create > bridge0 > # ifconfig bridge0 addm em0 > # ifconfig bridge0 addm lo1 > ifconfig: BRDGADD lo1: Invalid argument > > I can find one reference to this error message online from 2007: > > > https://lists.freebsd.org/pipermail/freebsd-net/2007-December/016102.html > > ... but this doesn't appear to be the problem I'm having (it seems as > though it was caused by an out-of-sync kernel and world, which I > obviously don't have). > > Any ideas how to get this working? > > M > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Sat Jun 25 19:24:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC47EB80A85 for ; Sat, 25 Jun 2016 19:24:04 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) Received: from black.birch.relay.mailchannels.net (black.birch.relay.mailchannels.net [23.83.209.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C06B16FE for ; Sat, 25 Jun 2016 19:24:03 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 0B9E71BC997; Sat, 25 Jun 2016 19:24:02 +0000 (UTC) Received: from bs3-dallas.accountservergroup.com (ip-10-21-3-36.us-west-2.compute.internal [10.21.3.36]) by relay.mailchannels.net (Postfix) with ESMTPA id 697661BC34D; Sat, 25 Jun 2016 19:24:01 +0000 (UTC) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from bs3-dallas.accountservergroup.com (bs3-dallas.accountservergroup.com [10.91.5.35]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.6.15); Sat, 25 Jun 2016 19:24:01 +0000 X-MC-Relay: Forwarding X-MailChannels-SenderId: _forwarded-from|212.69.61.187 X-MailChannels-Auth-Id: wwwh X-MC-Loop-Signature: 1466882641671:1726253306 X-MC-Ingress-Time: 1466882641671 Received: from cust187-dsl61.idnet.net ([212.69.61.187]:57476 helo=copperhead.int.arc7.info) by bs3-dallas.accountservergroup.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.87) (envelope-from ) id 1bGtBQ-000Fkc-Cl; Sat, 25 Jun 2016 14:24:00 -0500 Date: Sat, 25 Jun 2016 19:23:58 +0000 From: To: Marie Helene Kvello-Aune Cc: freebsd-net@freebsd.org Subject: Re: ifconfig: BRDGADD lo1: invalid argument Message-ID: <20160625192358.5365a68f@copperhead.int.arc7.info> In-Reply-To: References: <20160625164240.7cea7587@copperhead.int.arc7.info> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PopBeforeSMTPSenders: org.freebsd.security@io7m.com, io.github.lmax-exchange@io7m.com, com.the-blueprints@io7m.com, com.dropbox@io7m.com, com.rockstargames@io7m.com, org.openjdk@io7m.com, com.git-scm@io7m.com, com.bugsnag@io7m.com, com.jetbrains@io7m.com, com.apple@io7m.com, org.readium@io7m.com, com.google@io7m.com, com.slack@io7m.com, android-developers@io7m.com, com.skype@io7m.com, com.nexusmods@io7m.com, com.carpediemkravmaga@io7m.com, com.myfitnesspal@io7m.com, com.stronglifts@io7m.com, uk.co.discountsupplements@io7m.com, org.khanacademy@io7m.com, com.goodhempnutrition@io7m.com, org.freesound@io7m.com, org.mapdb@io7m.com, io.github.apitrace@io7m.com, org.codehaus@io7m.com, nu.xom@io7m.com, org.blender@io7m.com, org.jgrapht@io7m.com, org.eclipse@io7m.com, net.openvpn@io7m.com, org.apache.commons@io7m.com, de.jflex.users@io7m.com, org.mesa3d.mesa-users@io7m.com, net.java@io7m.com, com.io7m.lists@io7m.com, org.codehaus.mojo@io7m.com, com.meetup@io7m.com, org.archlinux@io7m.com, com.steampowered@io7m.com, com.blendswap@ io7m.com, org.opengl@io7m.com, legalandgeneral@io7m.com, org.freedesktop@io7m.com, org.jogamp@io7m.com, org.junit@io7m.com, org.apache.maven.user@io7m.com, org.sonatype@io7m.com, org.dyn4j@io7m.com, com.creative.opensource.openal@io7m.com, org.fossil-scm.fossil-users@io7m.com, github@io7m.com, code@io7m.com, contact@io7m.com, mark-ext@io7m.com, mark@io7m.com X-AuthUser: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 19:24:04 -0000 On 2016-06-25T19:16:06 +0000 Marie Helene Kvello-Aune wrote: > Check that lo1 has same MTU as bridge0. > > Regards, > Marie Helene Hello! Yes, I checked that (as one of the responses from the original thread suggested). Both lo1 and em0 (the real network adapter) have an MTU of 1500. M From owner-freebsd-net@freebsd.org Sat Jun 25 21:46:44 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9F77B82EC1 for ; Sat, 25 Jun 2016 21:46:44 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70FF71CFE for ; Sat, 25 Jun 2016 21:46:43 +0000 (UTC) (envelope-from zec@fer.hr) Received: from x23 (31.147.122.115) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.3.279.2; Sat, 25 Jun 2016 23:46:34 +0200 Date: Sat, 25 Jun 2016 23:46:36 +0200 From: Marko Zec To: CC: Subject: Re: ifconfig: BRDGADD lo1: invalid argument Message-ID: <20160625234636.2f086908@x23> In-Reply-To: <20160625164240.7cea7587@copperhead.int.arc7.info> References: <20160625164240.7cea7587@copperhead.int.arc7.info> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.1) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [31.147.122.115] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 21:46:45 -0000 On Sat, 25 Jun 2016 16:42:40 +0000 wrote: > Hello. > > I'm trying to create a bridge interface to isolate some jails on > private addresses. I'm on a near-pristine install of 10.3, updated to > 10.3-p5 via freebsd-update. > > The virtual interface to which the jails will be bound: > > # ifconfig lo1 create > > The bridge: > > # ifconfig bridge create > bridge0 > # ifconfig bridge0 addm em0 > # ifconfig bridge0 addm lo1 > ifconfig: BRDGADD lo1: Invalid argument if_bridge(4) works only with ethernet interfaces, and lo(4) isn't such a thing. Assuming you are using vnet jails, take a look at if_epair(4): assign one endpoint to the bridge, and the another one to the jail. If you're not using vnet jails, you should simply add an alias address to em0. > I can find one reference to this error message online from 2007: > > https://lists.freebsd.org/pipermail/freebsd-net/2007-December/016102.html > > ... but this doesn't appear to be the problem I'm having (it seems as > though it was caused by an out-of-sync kernel and world, which I > obviously don't have). > > Any ideas how to get this working? > > M > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sat Jun 25 22:10:11 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60083B8028C for ; Sat, 25 Jun 2016 22:10:11 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) Received: from nov-007-i540.relay.mailchannels.net (nov-007-i540.relay.mailchannels.net [46.232.183.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 752C91706 for ; Sat, 25 Jun 2016 22:10:08 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 169276015D for ; Sat, 25 Jun 2016 22:01:41 +0000 (UTC) Received: from bs3-dallas.accountservergroup.com (ip-10-21-3-36.us-west-2.compute.internal [10.21.3.36]) by relay.mailchannels.net (Postfix) with ESMTPA id 3E4836195E for ; Sat, 25 Jun 2016 22:01:40 +0000 (UTC) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from bs3-dallas.accountservergroup.com (bs3-dallas.accountservergroup.com [10.92.147.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.6.15); Sat, 25 Jun 2016 22:01:40 +0000 X-MC-Relay: Forwarding X-MailChannels-SenderId: _forwarded-from|212.69.61.187 X-MailChannels-Auth-Id: wwwh X-MC-Loop-Signature: 1466892132106:290862121 X-MC-Ingress-Time: 1466892132106 Received: from cust187-dsl61.idnet.net ([212.69.61.187]:65525 helo=copperhead.int.arc7.info) by bs3-dallas.accountservergroup.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.87) (envelope-from ) id 1bGvdz-000EgM-Bi for freebsd-net@freebsd.org; Sat, 25 Jun 2016 17:01:39 -0500 Date: Sat, 25 Jun 2016 22:01:37 +0000 From: To: freebsd-net@freebsd.org Subject: Filtering outbound traffic for private address jails? Message-ID: <20160625220137.1ed8de16@copperhead.int.arc7.info> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PopBeforeSMTPSenders: io.github.lmax-exchange@io7m.com, com.the-blueprints@io7m.com, com.dropbox@io7m.com, com.rockstargames@io7m.com, org.openjdk@io7m.com, com.git-scm@io7m.com, com.bugsnag@io7m.com, com.jetbrains@io7m.com, com.apple@io7m.com, org.readium@io7m.com, com.google@io7m.com, com.slack@io7m.com, android-developers@io7m.com, com.skype@io7m.com, com.nexusmods@io7m.com, com.carpediemkravmaga@io7m.com, com.myfitnesspal@io7m.com, com.stronglifts@io7m.com, uk.co.discountsupplements@io7m.com, org.khanacademy@io7m.com, com.goodhempnutrition@io7m.com, org.freesound@io7m.com, org.mapdb@io7m.com, io.github.apitrace@io7m.com, org.codehaus@io7m.com, nu.xom@io7m.com, org.blender@io7m.com, org.jgrapht@io7m.com, org.eclipse@io7m.com, net.openvpn@io7m.com, org.freebsd.security@io7m.com, org.apache.commons@io7m.com, de.jflex.users@io7m.com, org.mesa3d.mesa-users@io7m.com, net.java@io7m.com, com.io7m.lists@io7m.com, org.codehaus.mojo@io7m.com, com.meetup@io7m.com, org.archlinux@io7m.com, com.steampowered@io7m.com, com.blendswap@ io7m.com, org.opengl@io7m.com, legalandgeneral@io7m.com, org.freedesktop@io7m.com, org.jogamp@io7m.com, org.junit@io7m.com, org.apache.maven.user@io7m.com, org.sonatype@io7m.com, org.dyn4j@io7m.com, com.creative.opensource.openal@io7m.com, org.fossil-scm.fossil-users@io7m.com, github@io7m.com, code@io7m.com, contact@io7m.com, mark-ext@io7m.com, mark@io7m.com X-AuthUser: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 22:10:11 -0000 Hello. I have been searching for the best part of a day for a solution to this problem and quite frankly cannot believe that I've spent this long on something that appears to be so simple and that used to be fairly easy to achieve. Many years ago, I solved this problem on FreeBSD 6, but the way I did it there seems to no longer work on modern releases. The problem is this: I have a single public IP address. I want to run multiple jails. Back in the days of FreeBSD 6.*, the accepted way to do this seemed to be to create a new loopback device: # ifconfig lo1 create ... and then add a lot of private 127.0.0.* addresses, one per jail. Then, the real network adapter and the new loopback device were both added to a bridge (if_bridge). Unfortunately, I can't remember the exact details, but I believe that NAT was then enabled on the real interface. In order to filter traffic to, from, and between jails, pf rules were written that filtered the bridge device. This meant that jails could correctly send outbound traffic and receive responses (via pf states), could correctly receive specific inbound traffic (via rdr rules), and traffic in both directions could be filtered based on packets entering and leaving the bridge. However (see my other mailing list post), it seems that now with FreeBSD 10, you just can't add loopback devices to bridges. I can find no evidence of anyone online doing this, or even using the old bridge method that I just described! I can find one post in russian that seems to have the same error that I encounter, but nobody has any idea why it's happening. I can find dozens of blog posts describing how to set up jails on private IP addresses. They all follow the same pattern: 1. Create a loopback device. 2. Create a 127.0.0.* address on the loopback device. 3. Create a jail using the address you just added. 4. Set up pf and enable NAT between the real network adapter and the new loopback device. Unfortunately, at this point, you completely lose the ability to filter outbound jail traffic; All packets sent from a jail will obviously have their source address changed to that of the host and therefore it's not possible to distinguish between outbound host traffic and outbound jail traffic in filter rules. As far as I can tell, people are just not filtering outbound traffic, which seems insane! Is it really impossible to do this with FreeBSD 10? M From owner-freebsd-net@freebsd.org Sat Jun 25 23:17:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7AA57B815BA for ; Sat, 25 Jun 2016 23:17:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41A422032 for ; Sat, 25 Jun 2016 23:17:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id s66so159253717oif.1 for ; Sat, 25 Jun 2016 16:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=xRl4DRzKysmE3z1rItH/zcNEMIkygZ8oNgXo4JBDdaA=; b=vtEwtOW1mkNtu9s1CRTlSUMAeO2p4PQNT6iRfQaNvgfSlmDDr1MxxppIZ0i/0/T2Nl N1AO5e952vZiJMqFnhx3TsnkS3QScAbM8RmlUzEmfTQKl+aYxopg7l/eWJDcLvBJJEvG XbSHpphueITK8a2nc7kxzSsUX7fH4F9sTUoasJW4+tEajyXbFS2H1ve6W4KTGYqsuxX4 dgoNGgLFvmMLxd3vHKJvadbL7ZVx9zGFbn8KB99jiKWCfsLWgqIkYeYf5YiZoa5y5S+D 8WvlhkagZL/nusJH3F3E2FrC5tzlLU9z92HuXmVlU/xpBPZwl9GVRGtHOYLnKqftHr3K fPiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=xRl4DRzKysmE3z1rItH/zcNEMIkygZ8oNgXo4JBDdaA=; b=HDQuBzXZrEE2tsuP510QubU8XvKy1nHaVorn8DTFQBrXS6GSPweJNOsiLb3R2QMkRa BNtkP+Jmp7CyYUbtigfnR4btIVto8Uh9+DzQWdHdnCtzsYEywQwvCmmMvW33XJiVahI9 xFa/09USXAAbhvs/LEjthudncEi6gisGLqdoZQN6dxmipfYEvxVvJJ0TMiLN340To5WH ICV8+v/syI/Q9Kzkj1lLnIIts0Ywvaf+5IysmPJhWp84/vE7VM30cg3qSqzBBvriSuPS 4lpTDKhZTPP9C/GQTBN+O1Q6nC6kFicMQa/dE66uV3FVFswQWkTyKvD6zIhVU6CtCsp4 QsRg== X-Gm-Message-State: ALyK8tKQ666gQutkWfh5n6PwHkFWatYGaLp4vNWtRJ42o6SxeYPttI5QX4g3GUgij5RfL8WrPB8vLOR3ByKQCg== X-Received: by 10.202.224.136 with SMTP id x130mr7332384oig.105.1466896674312; Sat, 25 Jun 2016 16:17:54 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.202.168.149 with HTTP; Sat, 25 Jun 2016 16:17:53 -0700 (PDT) In-Reply-To: <20160625220137.1ed8de16@copperhead.int.arc7.info> References: <20160625220137.1ed8de16@copperhead.int.arc7.info> From: Alan Somers Date: Sat, 25 Jun 2016 17:17:53 -0600 X-Google-Sender-Auth: odxHu3bxjx0UoOmfCv427q9WCFA Message-ID: Subject: Re: Filtering outbound traffic for private address jails? To: org.freebsd.security@io7m.com Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 23:17:55 -0000 On Sat, Jun 25, 2016 at 4:01 PM, wrote: > Hello. > > I have been searching for the best part of a day for a solution to this > problem and quite frankly cannot believe that I've spent this long on > something that appears to be so simple and that used to be fairly easy > to achieve. Many years ago, I solved this problem on FreeBSD 6, but the > way I did it there seems to no longer work on modern releases. > > The problem is this: I have a single public IP address. I want to run > multiple jails. > > Back in the days of FreeBSD 6.*, the accepted way to do this seemed to > be to create a new loopback device: > > # ifconfig lo1 create > > ... and then add a lot of private 127.0.0.* addresses, one per jail. > > Then, the real network adapter and the new loopback device were both > added to a bridge (if_bridge). Unfortunately, I can't remember the exact > details, but I believe that NAT was then enabled on the real interface. > In order to filter traffic to, from, and between jails, pf rules were > written that filtered the bridge device. > > This meant that jails could correctly send outbound traffic and > receive responses (via pf states), could correctly receive specific > inbound traffic (via rdr rules), and traffic in both directions could be > filtered based on packets entering and leaving the bridge. > > However (see my other mailing list post), it seems that now with > FreeBSD 10, you just can't add loopback devices to bridges. I can find > no evidence of anyone online doing this, or even using the old bridge > method that I just described! I can find one post in russian that seems > to have the same error that I encounter, but nobody has any idea why > it's happening. > > I can find dozens of blog posts describing how to set up jails on > private IP addresses. They all follow the same pattern: > > 1. Create a loopback device. > 2. Create a 127.0.0.* address on the loopback device. > 3. Create a jail using the address you just added. > 4. Set up pf and enable NAT between the real network adapter and > the new loopback device. > > Unfortunately, at this point, you completely lose the ability to filter > outbound jail traffic; All packets sent from a jail will obviously have > their source address changed to that of the host and therefore it's not > possible to distinguish between outbound host traffic and outbound jail > traffic in filter rules. > > As far as I can tell, people are just not filtering outbound traffic, > which seems insane! > > Is it really impossible to do this with FreeBSD 10? > > M I'm filtering outbound traffic, but I'm not using NAT on the jail host. Instead, I have a dedicated router doing NAT, and my jail host has multiple IP addresses. At first I tried using traditional shared-address jails, but the firewall rules quickly got very complicated, especially for dealing with IPv6 and other non-IPv4 traffic. So I switched to using vimage jails. I use iocage to setup my jails, and pf to filter them. A simplified version of my pf.conf follows: www_services = "{ http, https, 8080 }" host_iface = "em0" dmz_iface = "em1" www_jail_iface = "vnet0:1" www_ip = "192.168.0.40" set state-policy if-bound scrub in block in all block out all pass in on $host_iface pass out on $host_iface set skip on lo0 # Allow all traffic to the DMZ. Filtering happens on individual vnet # interfaces pass in on $dmz_iface pass out on $dmz_iface # Put the www jail in a DMZ. Don't allow outgoing traffic from it except for # the webserver pass out on $www_jail_iface proto tcp to $www_ip port $www_services keep state # Uncomment next line to allow outbound traffice from www jail # pass in on $www_jail_iface -Alan From owner-freebsd-net@freebsd.org Sat Jun 25 23:21:49 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FF80B817C3 for ; Sat, 25 Jun 2016 23:21:49 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) Received: from khaki.cherry.relay.mailchannels.net (khaki.cherry.relay.mailchannels.net [23.83.223.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 518F02638 for ; Sat, 25 Jun 2016 23:21:45 +0000 (UTC) (envelope-from org.freebsd.security@io7m.com) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 19361100954; Sat, 25 Jun 2016 22:05:55 +0000 (UTC) Received: from bs3-dallas.accountservergroup.com (ip-10-35-137-113.us-west-2.compute.internal [10.35.137.113]) by relay.mailchannels.net (Postfix) with ESMTPA id 3B2FB100946; Sat, 25 Jun 2016 22:05:54 +0000 (UTC) X-Sender-Id: _forwarded-from|212.69.61.187 Received: from bs3-dallas.accountservergroup.com (bs3-dallas.accountservergroup.com [10.91.5.35]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.6.15); Sat, 25 Jun 2016 22:05:55 +0000 X-MC-Relay: Forwarding X-MailChannels-SenderId: _forwarded-from|212.69.61.187 X-MailChannels-Auth-Id: wwwh X-MC-Loop-Signature: 1466892386134:1352427866 X-MC-Ingress-Time: 1466892386134 Received: from cust187-dsl61.idnet.net ([212.69.61.187]:59860 helo=copperhead.int.arc7.info) by bs3-dallas.accountservergroup.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.87) (envelope-from ) id 1bGvi5-000FUG-5z; Sat, 25 Jun 2016 17:05:53 -0500 Date: Sat, 25 Jun 2016 22:05:51 +0000 From: To: Marko Zec Cc: Subject: Re: ifconfig: BRDGADD lo1: invalid argument Message-ID: <20160625220551.646eccb6@copperhead.int.arc7.info> In-Reply-To: <20160625234636.2f086908@x23> References: <20160625164240.7cea7587@copperhead.int.arc7.info> <20160625234636.2f086908@x23> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PopBeforeSMTPSenders: com.rockstargames@io7m.com, com.myfitnesspal@io7m.com, com.git-scm@io7m.com, org.codehaus@io7m.com, io.github.lmax-exchange@io7m.com, com.meetup@io7m.com, org.readium@io7m.com, org.khanacademy@io7m.com, com.nexusmods@io7m.com, io.github.apitrace@io7m.com, com.apple@io7m.com, org.apache.commons@io7m.com, org.apache.maven.user@io7m.com, org.freebsd.security@io7m.com, com.stronglifts@io7m.com, org.junit@io7m.com, com.carpediemkravmaga@io7m.com, legalandgeneral@io7m.com, uk.co.discountsupplements@io7m.com, com.the-blueprints@io7m.com, org.codehaus.mojo@io7m.com, com.skype@io7m.com, contact@io7m.com, com.goodhempnutrition@io7m.com, net.java@io7m.com, com.dropbox@io7m.com, com.io7m.lists@io7m.com, org.openjdk@io7m.com, org.jgrapht@io7m.com, com.google@io7m.com, android-developers@io7m.com, org.opengl@io7m.com, org.dyn4j@io7m.com, org.mapdb@io7m.com, com.jetbrains@io7m.com, org.eclipse@io7m.com, mark@io7m.com, com.slack@io7m.com, code@io7m.com, net.openvpn@io7m.com, nu.xom@io7m.com, github@io7m.com, de.jflex .users@io7m.com, com.blendswap@io7m.com, com.creative.opensource.openal@io7m.com, org.archlinux@io7m.com, org.jogamp@io7m.com, com.steampowered@io7m.com, mark-ext@io7m.com, org.mesa3d.mesa-users@io7m.com, org.sonatype@io7m.com, org.freesound@io7m.com, org.blender@io7m.com, org.freedesktop@io7m.com, com.bugsnag@io7m.com, org.fossil-scm.fossil-users@io7m.com X-AuthUser: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2016 23:21:49 -0000 Hello! On 2016-06-25T23:46:36 +0200 Marko Zec wrote: > > if_bridge(4) works only with ethernet interfaces, and lo(4) isn't such a > thing. Has this always been the case? I'm almost certain that I set up jails with extra loopback devices that communicated over bridges back in the FreeBSD 6 days. > Assuming you are using vnet jails, take a look at if_epair(4): assign > one endpoint to the bridge, and the another one to the jail. I'm not using vnet jails. I'm actually just trying to get filtering of outbound traffic (see the other mail I sent to this list a few seconds before you responded). > If you're not using vnet jails, you should simply add an alias address > to em0. Could you explain a little more here? M