From owner-freebsd-net@freebsd.org Sun Jun 25 14:54:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F3D1D92E64; Sun, 25 Jun 2017 14:54:31 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x233.google.com (mail-wr0-x233.google.com [IPv6:2a00:1450:400c:c0c::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE98E70DFE; Sun, 25 Jun 2017 14:54:30 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x233.google.com with SMTP id k67so121281522wrc.2; Sun, 25 Jun 2017 07:54:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=EHRT7r60qlKUbl2PrY0ILdGA/eiVC6MuO//XFBnYNFY=; b=nhRCcdYd0bXzBbNuvLETpqnQp++k4jsFSBahXn7hVe9bsivNQzXhVzvX7BGB5xl6yn v7vAxanVKqLWCp9w33AVHoBGrQG0O2rKYbzlRV2ZAWx67yNM5FbgcnDedLL5UzVZdDCO dEBkybR5l76jgjVnlPopui+OZYGTzhvwb0vKxtpzNaN880rastHlmnYFQ3jP0KC0zgcs BmZFhWaWE5rHXvps08qPO7JEWYLXQC4DM8NxheM7jg4DWPr+ADHyFnCrbUIJcQIQyIo3 hqrFouKJAOYCUTruG4hL2pdbi7oPoafNilCBAf9r7Er5sOOTYFZv3tNHBwT2fF6N4HPC CEVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=EHRT7r60qlKUbl2PrY0ILdGA/eiVC6MuO//XFBnYNFY=; b=ljQz57AVxJr3PVuAQNRbBwO0vuuUYMhAQALTyLVjaTtgT2WQEvX8vw/FuHWN07hU7t vPjB2jHD+1j6JjA4qCxGKOLlpUH8pzQEyzJ9oyPHEJlm6WWDx36Ng8FwNadSKeRSod+y QZGmZ5KyI66vczyaocQHMAeRLaAK8puBcISbrlXP2bJbUpAVrz1fBxprg/DKzpc5OKsR V512SHSvKivI3A7pfwrBb9Meg1KAWOOYeKSKEXKS8xPcF/LQbIGTvfbAxoCv6BzCykhD TkFAEBjc8x732+6Sdomcuy56xz0wueMYlt0PDcBs1Ur7PXSafgCbIkJmuQ0268SsOvvM 3Bmw== X-Gm-Message-State: AKS2vOzs4tB03/meVo+w3+uYrFdOH/5Es0Zlpxe9Pcf9KfI3jJSW/ao2 Y7x97tlyyx30GLmABbs= X-Received: by 10.223.144.39 with SMTP id h36mr11995373wrh.114.1498402467549; Sun, 25 Jun 2017 07:54:27 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id m73sm10541797wmi.25.2017.06.25.07.54.26 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 25 Jun 2017 07:54:26 -0700 (PDT) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> Date: Sun, 25 Jun 2017 16:54:25 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> To: FreeBSD Net , freebsd-scsi@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 14:54:31 -0000 > On 30 Dec 2016, at 22:55, Ben RUBSON wrote: >=20 > Hello, >=20 > 2 FreeBSD 11.0-p3 servers, one iSCSI initiator, one target. > Both with Mellanox ConnectX-3 40G. >=20 > Since a few days, sometimes, under undetermined circumstances, as soon = as there is some (very low) iSCSI traffic, some of the disks get = disconnected : > kernel: WARNING: 192.168.2.2 (iqn......): no ping reply (NOP-Out) = after 5 seconds; dropping connection >=20 > At the same moment, sysctl counters hw.mlxen1.stat.rx_ring*.error grow = on initiator side. >=20 > I then tried to reproduce these network errors burning the link at 40G = full-duplex using iPerf. > But I did not manage to increase these error counters. >=20 > It's strange because it's a sporadic issue, I can have traffic on = iSCSI disks without any issue, and sometimes, they get disconnected with = errors growing. > On 01 Jan 2017, at 09:16, Meny Yossefi wrote: >=20 > Any chance you ran out of mbufs in the system? > On 02 Jan 2017, at 12:09, Ben RUBSON wrote: >=20 > I think you are right, this could be a mbufs issue. > Here are some more numbers : >=20 > # vmstat -z | grep -v "0, 0$" > ITEM SIZE LIMIT USED FREE REQ = FAIL SLEEP > 4 Bucket: 32, 0, 2673, 28327, 88449799, = 17317, 0 > 8 Bucket: 64, 0, 449, 15609, 13926386, = 4871, 0 > 12 Bucket: 96, 0, 335, 5323, 10293892, = 142872, 0 > 16 Bucket: 128, 0, 533, 6070, 7618615, = 472647, 0 > 32 Bucket: 256, 0, 8317, 22133, 36020376, = 563479, 0 > 64 Bucket: 512, 0, 1238, 3298, 20138111, = 11430742, 0 > 128 Bucket: 1024, 0, 1865, 2963, 21162182, = 158752, 0 > 256 Bucket: 2048, 0, 1626, 450, 80253784, = 4890164, 0 > mbuf_jumbo_9k: 9216, 603712, 16400, 8744, 4128521064, = 2661, 0 > On 03 Jan 2017, at 07:27, Meny Yossefi wrote: >=20 > Have you tried increasing the mbufs limit?=20 > (sysctl) kern.ipc.nmbufs (Maximum number of mbufs allowed) > On 04 Jan 2017, at 14:47, Ben RUBSON wrote: >=20 > No I did not try this yet. > However, from the numbers above (and below), I think I should increase = kern.ipc.nmbjumbo9 instead ? > On 30 Jan 2017, at 15:36, Ben RUBSON wrote: >=20 > So, to give some news, increasing kern.ipc.nmbjumbo9 helped a lot. > Just a very little issue (compared to the others before) over the last = 3 weeks. Hello, I'm back today with this issue. Above is my discussion with Meny from Mellanox at the beginning of 2017. (topic was "iSCSI failing, MLX rx_ring errors ?", on freebsd-net list) So this morning issue came again, some of my iSCSI disks were = disconnected. Below are some numbers. # vmstat -z | grep -v "0, 0$" ITEM SIZE LIMIT USED FREE REQ FAIL = SLEEP 8 Bucket: 64, 0, 654, 8522, 28604967, 11, 0 12 Bucket: 96, 0, 976, 5092, 23758734, 78, 0 32 Bucket: 256, 0, 789, 4491, 43446969, 137, 0 64 Bucket: 512, 0, 666, 2750, 47568959, 1272018, 0 128 Bucket: 1024, 0, 1047, 1249, 28774042, 232504, 0 256 Bucket: 2048, 0, 1611, 369, 139988097, 8931139, 0 vmem btag: 56, 0, 2949738, 15506, 18092235, 20908, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 8776, 8610737115, 297, 0 # uname -rs FreeBSD 11.0-RELEASE-p8 # uptime 3:34p.m. up 88 days, 15:57, 2 users, load averages: 0.95, 0.67, 0.62 # grep kern.ipc.nmb /boot/loader.conf=20 kern.ipc.nmbjumbo9=3D2037529 kern.ipc.nmbjumbo16=3D1 # sysctl kern.ipc | grep mb kern.ipc.nmbufs: 26080380 kern.ipc.nmbjumbo16: 4 kern.ipc.nmbjumbo9: 6112587 kern.ipc.nmbjumbop: 2037529 kern.ipc.nmbclusters: 4075060 kern.ipc.maxmbufmem: 33382887424 # ifconfig mlxen1 mlxen1: flags=3D8843 metric 0 = mtu 9020 = options=3Ded07bb nd6 options=3D29 media: Ethernet autoselect (40Gbase-CR4 ) status: active I just caught the issue growing : # vmstat -z | grep mbuf_jumbo_9k ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP mbuf_jumbo_9k: 9216, 2037529, 16415, 7316,8735246407, 665, 0 mbuf_jumbo_9k: 9216, 2037529, 16411, 7320,8735286748, 665, 0 mbuf_jumbo_9k: 9216, 2037529, 16415, 7316,8735298937, 667, 0 mbuf_jumbo_9k: 9216, 2037529, 16438, 7293,8735337634, 667, 0 mbuf_jumbo_9k: 9216, 2037529, 16407, 7324,8735354339, 668, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735382105, 669, 0 mbuf_jumbo_9k: 9216, 2037529, 16402, 7329,8735392836, 671, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735423910, 671, 0 mbuf_jumbo_9k: 9216, 2037529, 16415, 7316,8735456393, 671, 0 mbuf_jumbo_9k: 9216, 2037529, 16409, 7322,8735472284, 672, 0 mbuf_jumbo_9k: 9216, 2037529, 16420, 7311,8735512237, 673, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735518502, 675, 0 mbuf_jumbo_9k: 9216, 2037529, 16410, 7321,8735543668, 676, 0 mbuf_jumbo_9k: 9216, 2037529, 16405, 7326,8735555646, 678, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735568986, 679, 0 mbuf_jumbo_9k: 9216, 2037529, 16414, 7317,8735579075, 680, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735603983, 681, 0 mbuf_jumbo_9k: 9216, 2037529, 16402, 7329,8735634273, 681, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735646057, 683, 0 mbuf_jumbo_9k: 9216, 2037529, 16402, 7329,8735658213, 684, 0 mbuf_jumbo_9k: 9216, 2037529, 16414, 7317,8735675678, 686, 0 mbuf_jumbo_9k: 9216, 2037529, 16415, 7316,8735686017, 687, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8735707335, 687, 0 mbuf_jumbo_9k: 9216, 2037529, 16414, 7317,8736016546, 708, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8736037292, 709, 0 mbuf_jumbo_9k: 9216, 2037529, 16405, 7326,8736053865, 710, 0 mbuf_jumbo_9k: 9216, 2037529, 16402, 7329,8736070103, 711, 0 mbuf_jumbo_9k: 9216, 2037529, 16407, 7324,8736086810, 711, 0 mbuf_jumbo_9k: 9216, 2037529, 16430, 7301,8736098568, 713, 0 mbuf_jumbo_9k: 9216, 2037529, 16405, 7326,8736122803, 714, 0 mbuf_jumbo_9k: 9216, 2037529, 16417, 7314,8736134322, 715, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8736152338, 715, 0 mbuf_jumbo_9k: 9216, 2037529, 16403, 7328,8736167677, 715, 0 mbuf_jumbo_9k: 9216, 2037529, 16400, 7331,8736170783, 717, 0 mbuf_jumbo_9k: 9216, 2037529, 16445, 7286,8736546084, 733, 0 During this, top was reporting the following : Mem: 4056K Active, 426M Inact, 59G Wired, 2531M Free And in /var/log/messages : kernel: WARNING: 192.168.2.2 (iqn......): no ping reply (NOP-Out) after = 5 seconds; dropping connection Any idea why I'm experiencing this ? Thank you very much for your help & support, Best regards, Ben From owner-freebsd-net@freebsd.org Sun Jun 25 15:14:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C784D938F3; Sun, 25 Jun 2017 15:14:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46486730B7; Sun, 25 Jun 2017 15:14:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-it0-x236.google.com with SMTP id m84so12725525ita.0; Sun, 25 Jun 2017 08:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0Y33zcri/NfWAWZUmPaK9x6nbkgvzR6xwBm4MjKLF5k=; b=XSRnqiUz2Qn0zjXkUE6lGWl8qzumHijXNtl2mphiybXwUuHPnztgRSy8btsxq/0xaP +bgCmVze/74NaiT6d5PCqyaDPPEsILNIrhabOXdKAmsfgtlSbJFw0muugt9P8KIl0b1R F19L15mkg57K0+x0GE2F1aZgGIH/UmztIFICvmz80RR8f6cZoajSizoszLC9Qw4l+w6z fdP9LzVkgTrGo1+25gABG34NymosYB9MtHgcNqq79ZHVT/AwmTX6mSvUZKZFPEwWjSbv LAJnZyxPT0759q6upU8YJ7nZV6YYAk9AEbgtrXG/wZR5a/doIp/y1WxmiU4d+WARue0S oqoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0Y33zcri/NfWAWZUmPaK9x6nbkgvzR6xwBm4MjKLF5k=; b=iDnDzpBDIjaGPtaWLSwDMnwRZQGs2tifb4i2gv58peoqD8JAUR6uZe5L4TPuPgpKcC uJPCW7dxb4OccWdHxyOKW4/DyN+I8s/ehAV5K7s1F2yvtIv2JwhXkJKqlSyhVP28noUk DQ+7kD/J3nXZUrL4qE4HYK6DsialF+990eDDkHc5475jtH6YkgYKDMKHWcj7VtC7B0Yc C2hh37x9VxEFyzBQDL0NmIRQR1oDAjS09HUUzYaFJcEvCh5ea/WR0X4B6BlMoThgiU2X t1n9Lg0cNu+MspNF0QoDeNzcbjosLtVwi6RE+5kanmCGxMz1iZW/AOzlOo7hNDK6KD35 HnPg== X-Gm-Message-State: AKS2vOw0NyPb+yHULPnN1jkHORu5LQ8Eb3xKzCxtcsm3wC64T2d5Hv/6 EHUJIwkuwfHy1ENu/Fsi+ajTqZDaLQ== X-Received: by 10.36.254.134 with SMTP id w128mr6216777ith.2.1498403647595; Sun, 25 Jun 2017 08:14:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.156.21 with HTTP; Sun, 25 Jun 2017 08:14:07 -0700 (PDT) In-Reply-To: <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> From: Ryan Stone Date: Sun, 25 Jun 2017 11:14:07 -0400 Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 15:14:08 -0000 Is this setup using the mlx4_en driver? If so, recent versions of that driver has a regression when using MTUs greater than the page size (4096 on i386/amd64). The bug will cause the card to drop packets when the system is under memory pressure, and in certain causes the card can get into a state when it is no longer able to receive packets. I am working on a fix; I can post a patch when it's complete. From owner-freebsd-net@freebsd.org Sun Jun 25 15:28:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70BE7D93D9E; Sun, 25 Jun 2017 15:28:35 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x242.google.com (mail-wr0-x242.google.com [IPv6:2a00:1450:400c:c0c::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 138E373842; Sun, 25 Jun 2017 15:28:34 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x242.google.com with SMTP id 77so24896158wrb.3; Sun, 25 Jun 2017 08:28:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=N1VzSMP7GTIsjlChBfgT/1wq+KUsh8jOhru8CYgipyE=; b=Xm5XC7/G6JyInbRznqfLTEnabQYYxipTSoRcb84BMNBq7y/8GxdBOl9dal9Jr4V5a3 y6/OSkBjUKUQBN6b4BNiUf+Gm4ctsHgJrjgoDup17KXzWinU07A2jj8CET8o/l3Joy6t d6vS5tywKM4bHmofeGikK+gptWyVd1KaoUfVsEhk00ENDUyjFN6sdjadbSV0xo//SzLr FVtrme+iUhXMIZxXYxk1dIgDlHhnYOXAlRy8FK/vtLPf/qQAcWC9570WJybZu9oOx5pp /aNZ0+wq7yNkdoYpiC3+XX+n8BdACk/tqi3KqIdeEf0QmYtTarDYKPFZB34EeKh4o8EU Dyxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=N1VzSMP7GTIsjlChBfgT/1wq+KUsh8jOhru8CYgipyE=; b=LZwlA9tzGAHfF/x3dYFVWlpzoHhTZSzsfuvegKHUJl+r83p0NL3yqDQ4B8PEYYJezW emRh02m4EiblSyJyM2eFUbn8hxLZW3gx6JJISt6hjPhx55YdsW5BIwmCCBOCogu+d9tc +lylFwcnoRBdjmsPeEY1ggvSwI3LygtkT2c1hCNcBWeVSvYjILWq+irhzj8bKb324Nt5 PdBvZVJedInFCAEe0RZe2ijknzTVjlEzWxhFxPb3e//lYADDFzV0LCGoapJ8tr8wqVVN 5IiVv0pd/PyFhvVnT+7hNUq1DHBV+oETr2eakdjCHtUJzDw2g5jgGLONVvLTxcS+EX3j adZQ== X-Gm-Message-State: AKS2vOznj4kNtJM6wvH72KAAxDf4ghUHJJVWKMyPZEecwwQIk7Ev8r1m BTusItBYPGH+K9UDrxo= X-Received: by 10.28.10.194 with SMTP id 185mr10363386wmk.119.1498404513492; Sun, 25 Jun 2017 08:28:33 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id o6sm6159918wrc.48.2017.06.25.08.28.32 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 25 Jun 2017 08:28:32 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: Date: Sun, 25 Jun 2017 17:28:32 +0200 Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> To: Ryan Stone X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 15:28:35 -0000 > On 25 Jun 2017, at 17:14, Ryan Stone wrote: >=20 > Is this setup using the mlx4_en driver? If so, recent versions of = that driver has a regression when using MTUs greater than the page size = (4096 on i386/amd64). The bug will cause the card to drop packets when = the system is under memory pressure, and in certain causes the card can = get into a state when it is no longer able to receive packets. I am = working on a fix; I can post a patch when it's complete. Thank you very much for your feedback Ryan. Yes, my system is using mlx4_en driver, the one directly from FreeBSD = 11.0 sources tree. Any indicator I could catch to be sure I'm experiencing the issue you = are working on ? Sounds like anyway I may be suffering from it... Of course I would be glad to help testing your patch when it's complete. Thank you again, Ben From owner-freebsd-net@freebsd.org Sun Jun 25 15:32:01 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8FDAD94000; Sun, 25 Jun 2017 15:32:01 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACBB273B81; Sun, 25 Jun 2017 15:32:01 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-it0-x229.google.com with SMTP id b205so28631718itg.1; Sun, 25 Jun 2017 08:32:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=MsAEoNurtT5IYa40c+cR50mA6+sUo55yg6jG04sP4Ik=; b=ZYDBYSw2SSa7lX/Aisw5huwcTxYk9wG5H2lSjTbX5edRxEUaSiSx9FJ6Fd7ZMMMlIe EwOh9cnjpPC6pYaHp8fiBbe2h87uO9C409kb86TSAlS5R1NvUwZZTX4mbD9g7w8r4yd/ HPQnRDVpLNVn67cCYIs8j7u4t/8MHsh01DosYi57tEOO3E3AXdooTWuyinRSGrrXcU9T Wm5FGbg8Ih+oyQ4TJVnswxE6rpUbrY1+geQ4VRNHeqBfnRZ7C+QX4i8ksiy6vL4ENsTP qHboF5RrEj+XqvC9gP3eyq/ti8VzZJntKPkR6y0LZRYKis44jUEJs/KBay+JuqmHr3ts 11nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=MsAEoNurtT5IYa40c+cR50mA6+sUo55yg6jG04sP4Ik=; b=tZzX5ya7qG2x3CoovlCTt7CLw/I+7EGfzB5IBh8ZJgh7V68Pypz2d+xYQDSR3l6iuK 6jRLb5xPUVoQj8IORBRadS+/fqswZ4bHR0x1Sf0lWrq03WlaPwaccUnw8NDtAxS5HmeU HOBL8SYYDDto7uPdlf/7mJBPScRncAjE+pC3COWZqcE1V4KvwLNmFCl7XAdB/nS377Ws /X0DpmEtS4G+0E2n07B28JvGdzWksSI4TO1P9m22SDId6z6DFRCulJbxmqpBiav+s22p 509UcXDmPj2BtWi63WFTnujEtHQDY8Tz7b+Q/KUmeNQbjro87KmN0DGftewh6isKXIay jtpQ== X-Gm-Message-State: AKS2vOzaagOaGSkhSHqrbEIyx+rgHxgkgzF0IW0NSeTipuONBwjwt1Ij woN7ZpcOZw11ajOg96CBa/H9hO9oDQ== X-Received: by 10.36.65.23 with SMTP id x23mr18161797ita.2.1498404721102; Sun, 25 Jun 2017 08:32:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.156.21 with HTTP; Sun, 25 Jun 2017 08:32:00 -0700 (PDT) In-Reply-To: References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> From: Ryan Stone Date: Sun, 25 Jun 2017 11:32:00 -0400 Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 15:32:02 -0000 Having looking at the original email more closely, I see that you showed an mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf clusters increase while you are far below the zone's limit means that you're definitely running into the bug I'm describing, and this bug could plausibly cause the iSCSI errors that you describe. The issue is that the newer version of the driver tries to allocate a single buffer to accommodate an MTU-sized packet. Over time, however, memory will become fragmented and eventually it can become impossible to allocate a 9k physically contiguous buffer. When this happens the driver is unable to allocate buffers to receive packets and is forced to drop them. Presumably, if iSCSI suffers too many packet drops it will terminate the connection. The older version of the driver limited itself to page-sized buffers, so it was immune to issues with memory fragmentation. From owner-freebsd-net@freebsd.org Sun Jun 25 16:56:49 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2FB0D9571C; Sun, 25 Jun 2017 16:56:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5623376166; Sun, 25 Jun 2017 16:56:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22e.google.com with SMTP id 77so124109801wrb.1; Sun, 25 Jun 2017 09:56:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3FL9L7x3JW4a1K9f2uyaxkqNrwNaS2oDAJKRyxGJF3s=; b=rVN+UHhTK5C1k0MsRtveBnlLadwcmlKbZ393jeaHwLbDmePZ1uE07tdb9gywHyLJqS qMe4d1jpjHznV4Ape8xyWHVgvJicKXdGOIIECgmnk6MDb85B4C1dK5qPIZ49ix97eGhG PtPONyMcEyD4PUHH3QwHC8JKeZJLut3eepdmoPxihKpYFF1jsaLCttgBp7+Y5c+8MSJd D1V1GeTfw1V71X9+M1beX9M1MaHhzq0vf2jD1QgX8+5DZXS1aV0k4Oef4mvn9O4I1vVl trAGHZ0T3NnSDCzDqGXLGBiu15v3nTELAjl6QN9R001IzWxCaJyhfLIiWk1L4Azs9Jvf D3gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3FL9L7x3JW4a1K9f2uyaxkqNrwNaS2oDAJKRyxGJF3s=; b=qGJeObOA7sHXZz6O63CsXaa4zTG56G0LvIliYgKPagQn8r23vdM1TTMC1Y2XMPv3tm gw2w2NvC2WhTbDB5jdvqm34TmlMLN+n55R+Ih9latd9mKEFm0vdKsLne3fWVKgsoHG4E 5VMhijRpkfcySGHG3nsAwwZADqjCf/U4mockEpGVtVpG9YVEzcVP/KF+S8h5MiULb6Vs 96/0iHVkXYKDSh4Z4bHWHI55WlU4LHKmDeuo/9Sibwhmfkv/0fBab1Cdc4Ck8qFyDXhw JGzKRkNV8/TdqLEi8G1boeRTyg/Na8Z8wtISf5D3lS+npSPCl1SgfwHA4O8yDC4jAo3C hWCw== X-Gm-Message-State: AKS2vOwvEi/bsm3JwNMotn+nmuz54683rTtA4wE8ZogwzA3HIxoU1RbT zkV1+cL+JEGvlw== X-Received: by 10.223.176.253 with SMTP id j58mr10795933wra.65.1498409807388; Sun, 25 Jun 2017 09:56:47 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id m26sm14755941wrm.4.2017.06.25.09.56.46 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 25 Jun 2017 09:56:46 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: Date: Sun, 25 Jun 2017 18:56:45 +0200 Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> To: Ryan Stone X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 16:56:49 -0000 > On 25 Jun 2017, at 17:32, Ryan Stone wrote: >=20 > Having looking at the original email more closely, I see that you = showed an mlxen interface with a 9020 MTU. Seeing allocation failures = of 9k mbuf clusters increase while you are far below the zone's limit = means that you're definitely running into the bug I'm describing, and = this bug could plausibly cause the iSCSI errors that you describe. >=20 > The issue is that the newer version of the driver tries to allocate a = single buffer to accommodate an MTU-sized packet. Over time, however, = memory will become fragmented and eventually it can become impossible to = allocate a 9k physically contiguous buffer. When this happens the = driver is unable to allocate buffers to receive packets and is forced to = drop them. Presumably, if iSCSI suffers too many packet drops it will = terminate the connection. The older version of the driver limited = itself to page-sized buffers, so it was immune to issues with memory = fragmentation. Thank you for your explanation Ryan. You say "over time", and you're right, I have to wait several days (here = 88) before the problem occurs. Strange however that in 2500MB free memory system is unable to find 9k = physically contiguous. But we never know :) Let's then wait for your patch ! (and reboot for now) Many thx ! Ben= From owner-freebsd-net@freebsd.org Sun Jun 25 21:00:03 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7A30D99E83 for ; Sun, 25 Jun 2017 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD9CE7D6A5 for ; Sun, 25 Jun 2017 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5PL01ej013591 for ; Sun, 25 Jun 2017 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201706252100.v5PL01ej013591@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-net@FreeBSD.org Subject: Problem reports for freebsd-net@FreeBSD.org that need special attention Date: Sun, 25 Jun 2017 21:00:03 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 21:00:03 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 165622 | [ndis][panic][patch] Unregistered use of FPU in k In Progress | 206581 | bxe_ioctl_nvram handler is faulty New | 204438 | setsockopt() handling of kern.ipc.maxsockbuf limi New | 205592 | TCP processing in IPSec causes kernel panic New | 206053 | kqueue support code of netmap causes panic New | 213410 | [carp] service netif restart causes hang only whe New | 215874 | [patch] [icmp] [mbuf_tags] teach icmp_error() opt New | 217748 | sys/dev/ixgbe/if_ix.c: PVS-Studio: Assignment to Open | 173444 | socket: IPV6_USE_MIN_MTU and TCP is broken Open | 193452 | Dell PowerEdge 210 II -- Kernel panic bce (broadc Open | 194485 | Userland cannot add IPv6 prefix routes Open | 194515 | Fatal Trap 12 Kernel with vimage Open | 199136 | [if_tap] Added down_on_close sysctl variable to t Open | 202510 | [CARP] advertisements sourced from CARP IP cause Open | 206544 | sendmsg(2) (sendto(2) too?) can fail with EINVAL; Open | 211031 | [panic] in ng_uncallout when argument is NULL Open | 211962 | bxe driver queue soft hangs and flooding tx_soft_ Open | 218653 | Intel e1000 network link drops under high network 18 problems total for which you should take action. From owner-freebsd-net@freebsd.org Mon Jun 26 00:20:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25E26D9D347 for ; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0CFD082C21 for ; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0933CD9D346; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07079D9D345 for ; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC3BD82C1F for ; Mon, 26 Jun 2017 00:20:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5Q0K3VV098417 for ; Mon, 26 Jun 2017 00:20:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 200382] Loading netgraph via bsnmpd, etc can cause domain to be registered after domain_finalize has been called Date: Mon, 26 Jun 2017 00:20:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 00:20:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D200382 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|net@FreeBSD.org |freebsd-net@FreeBSD.org --- Comment #3 from Mark Linimon --- Canonicalize assignment. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 26 00:20:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4CDC8D9D349 for ; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3AF4282C22 for ; Mon, 26 Jun 2017 00:20:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5Q0K3VZ098417 for ; Mon, 26 Jun 2017 00:20:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 200382] Loading netgraph via bsnmpd, etc can cause domain to be registered after domain_finalize has been called Date: Mon, 26 Jun 2017 00:20:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 00:20:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D200382 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|net@FreeBSD.org |freebsd-net@FreeBSD.org --- Comment #3 from Mark Linimon --- Canonicalize assignment. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 26 02:45:42 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30CE8D9F6E9 for ; Mon, 26 Jun 2017 02:45:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E9311AF4 for ; Mon, 26 Jun 2017 02:45:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5Q2jf9A083801 for ; Mon, 26 Jun 2017 02:45:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220076] [patch] [panic] [netgraph] repeatable kernel panic due to a race in ng_iface(4) Date: Mon, 26 Jun 2017 02:45:41 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 02:45:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220076 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- CC|freebsd-net@FreeBSD.org | Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jun 26 13:02:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D257D86471; Mon, 26 Jun 2017 13:02:10 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-vk0-x230.google.com (mail-vk0-x230.google.com [IPv6:2607:f8b0:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1D40783BF; Mon, 26 Jun 2017 13:02:09 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-vk0-x230.google.com with SMTP id 191so319311vko.2; Mon, 26 Jun 2017 06:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=jmEyFFLQSYeM6xEEdWTEN7u+GglUoER+WvbyAu6RKCY=; b=dDvce0nc7uS/JU9udWXQ7W9S76X6QrhtZrZSAqpnEA7RKlo6lnuXNGoG6glYfGd3k4 hdiR47Kf51r3abXe/SuQG8fbklEhugvrVXnOHJ8XDciWqMPWcbK1zi3sx2guZE2nEfKC KEvalK6kwd4gUXBZfF3+ZOrg2MoNElgjRSJ18tloglEfLzABw/80GOdf8Y/MBxmzZYA4 h59lzEeGIff/4QVR469hjAHBknuiGhuA3SCuFWrs6Mm7rS9YoHiaaWvw+d4Y7vrPSCra m8EcDdrv7oCLXD4aQ44pu2nKKH1jvyh44u7fc7F1L5ubTi1GJA5WrGgwTG9xub+q+uUh mJYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=jmEyFFLQSYeM6xEEdWTEN7u+GglUoER+WvbyAu6RKCY=; b=OpwFFiLa99YfU11FpYER1ZPCHXrUwSgylXiPb5PJjyoNG+gz0F20DXXOm39/ub1ZWs Rc4B0p0dVCuDg3/LI0Vu9symqeGF7a7v2EssyvrgarMfxn3i37TgbLiQsEOc2WtKiwgj 8bsnDJczeryG4loGPARdKhHv/UsFdisesMMcZ/BuDc/QpDV6aL4KkEPwujNVR34zv+73 +NIaD68RUw9hjUH7910MfHGtx8W3MokbKv2zrXfQ5hLmDRDJ5aQe/q1awrRqoDS9+m40 JlpKYrt2wD17+iqrtSYEY+XZsHLk/6AwJqIeKR9tvedC250PzHW62PGWEEBHqmwEFNss xrHg== X-Gm-Message-State: AKS2vOy7gT2SeyTpV0hNhYxqr1fae0A7+t3kVElb6vrNhWhmTz9C9GV2 owh5XTKwovq87NRMnTQsoAZP9Zcezg== X-Received: by 10.31.222.193 with SMTP id v184mr32122vkg.73.1498482129070; Mon, 26 Jun 2017 06:02:09 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.176.83.198 with HTTP; Mon, 26 Jun 2017 06:02:08 -0700 (PDT) In-Reply-To: References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> From: Edward Napierala Date: Mon, 26 Jun 2017 14:02:08 +0100 X-Google-Sender-Auth: oGGxrb1a2EIfNU6zTD5eQv8QPbk Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ryan Stone Cc: Ben RUBSON , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:02:10 -0000 2017-06-25 16:32 GMT+01:00 Ryan Stone : > Having looking at the original email more closely, I see that you showed an > mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > clusters increase while you are far below the zone's limit means that > you're definitely running into the bug I'm describing, and this bug could > plausibly cause the iSCSI errors that you describe. > > The issue is that the newer version of the driver tries to allocate a > single buffer to accommodate an MTU-sized packet. Over time, however, > memory will become fragmented and eventually it can become impossible to > allocate a 9k physically contiguous buffer. When this happens the driver > is unable to allocate buffers to receive packets and is forced to drop > them. Presumably, if iSCSI suffers too many packet drops it will terminate > the connection. [..] More specifically, it will terminate the connection when there's no "ping reply" from the other side for the configured amount of time, which defaults to five seconds. It can be changed using the kern.iscsi.ping_timeout sysctl, as described in iscsi(4). From owner-freebsd-net@freebsd.org Mon Jun 26 13:16:23 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CACBD86CDA; Mon, 26 Jun 2017 13:16:23 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward1m.cmail.yandex.net (forward1m.cmail.yandex.net [5.255.216.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A21C378EB1; Mon, 26 Jun 2017 13:16:22 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1o.mail.yandex.net (smtp1o.mail.yandex.net [37.140.190.26]) by forward1m.cmail.yandex.net (Yandex) with ESMTP id 3B4B3215F8; Mon, 26 Jun 2017 16:16:13 +0300 (MSK) Received: from smtp1o.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp1o.mail.yandex.net (Yandex) with ESMTP id 94CEC1300B9E; Mon, 26 Jun 2017 16:16:11 +0300 (MSK) Received: by smtp1o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id VSmE86ptau-GAcSd94F; Mon, 26 Jun 2017 16:16:10 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1498482970; bh=UwoC2WoPuBeLK+HvGWTll0M+Zmas0TUMuu8l5xHy7vY=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=MPpPn9vIZU0GezIh67TJPsoGiCPwKKFVMJnVHXHl8jaZm8rvEoU83F1u6bYJXu+d4 3ATLDqm/fwSkAnNaowxhdQGyieSSlRfqihokaKov36noijxH6iHd5C7yImLSvQelU/ BrFrFwD4FYY6eqPT5zutCCAWRn9EIEdu9PHiW68E= Authentication-Results: smtp1o.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0,1 0,1 0 Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ryan Stone , Ben RUBSON Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> Date: Mon, 26 Jun 2017 16:13:33 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="smqF3QFS4WoFbpvK6WMrFmhjvPMtBWeiE" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:16:23 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --smqF3QFS4WoFbpvK6WMrFmhjvPMtBWeiE Content-Type: multipart/mixed; boundary="LXCCDuFkvV5DOvwWRd8qkkXQ6ssJnmhDg"; protected-headers="v1" From: "Andrey V. Elsukov" To: Ryan Stone , Ben RUBSON Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Message-ID: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> Subject: Re: mbuf_jumbo_9k & iSCSI failing References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> In-Reply-To: --LXCCDuFkvV5DOvwWRd8qkkXQ6ssJnmhDg Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 25.06.2017 18:32, Ryan Stone wrote: > Having looking at the original email more closely, I see that you showe= d an > mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf= > clusters increase while you are far below the zone's limit means that > you're definitely running into the bug I'm describing, and this bug cou= ld > plausibly cause the iSCSI errors that you describe. >=20 > The issue is that the newer version of the driver tries to allocate a > single buffer to accommodate an MTU-sized packet. Over time, however, > memory will become fragmented and eventually it can become impossible t= o > allocate a 9k physically contiguous buffer. When this happens the driv= er > is unable to allocate buffers to receive packets and is forced to drop > them. Presumably, if iSCSI suffers too many packet drops it will termi= nate > the connection. The older version of the driver limited itself to > page-sized buffers, so it was immune to issues with memory fragmentatio= n. I think it is not mlxen specific problem, we have the same symptoms with ixgbe(4) driver too. To avoid the problem we have patches that are disable using of 9k mbufs, and instead only use 4k mbufs. --=20 WBR, Andrey V. Elsukov --LXCCDuFkvV5DOvwWRd8qkkXQ6ssJnmhDg-- --smqF3QFS4WoFbpvK6WMrFmhjvPMtBWeiE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAllRCH0ACgkQAcXqBBDI oXobwAf+Lxnto9w+KaTuAIkktZnoN42kjUNtY1bMNFn5amhEgFNlUeQxqFymZdvQ h7f2cAiNJ3SLVxiw/cxbeOK4r2Bk53JwNBEwI/VtyNCDLyikJ7Ov8yVOAgG4RydA llp+ZXsko19zxdlR5aBW140egiRCXTMvbNZ4IoqE3GiHwS6TDEFglbbEbUJK0r9l A9zlW+0EAxo3UELSQhfymIALfXHCPLzM0AYf/VvWdvNIio3Y1ZMeZK6Rkofgefux yddJnL54pVWKz1LvfuSEtE+wW9Tm/nF2MeRrdLKsiSWJGuUwTKlMq4porJ3K0Viq V6MHyi2U7818TQcDBbOkwrztEL9l3w== =S8WI -----END PGP SIGNATURE----- --smqF3QFS4WoFbpvK6WMrFmhjvPMtBWeiE-- From owner-freebsd-net@freebsd.org Mon Jun 26 13:27:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02E82D87071; Mon, 26 Jun 2017 13:27:12 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22c.google.com (mail-wr0-x22c.google.com [IPv6:2a00:1450:400c:c0c::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 936FF79506; Mon, 26 Jun 2017 13:27:11 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22c.google.com with SMTP id 77so144753916wrb.1; Mon, 26 Jun 2017 06:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=L/8RwOIW8TnkqeTb/juk5I5r3hOn5WAsY5Ym+odT6Qw=; b=VzOnwldmkuC6wbBYbuQ3krqYJtm3fkOtvj56d0D/jgR6+rRnCKZFd8PnAbh7DJ3SA4 6CpPImnK5xeVdLTpC4xeT4VCgCVfKUYzp/jNbSr1FwxvEiQ6+KmsBaSQHCY/oPmNBgE0 Brt7806mZqXV65FANpOOKX0rJM6QF2dhvP+87Rm8BGhT7rf31foYXk16mRqJbujjU0wP 5/qQK3k8TsLgyIlF3gzlon+UEbrONpV2WZ9tZcNzIUlx1mkwXXtneyN8TZ07SIi/2kx6 6f+2YVTFUtCwtDfb79Zu589mUozBRfUEyofIZd8KKSfRRmEwXBTf8cZxNyTzYVlE6MJu rmPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=L/8RwOIW8TnkqeTb/juk5I5r3hOn5WAsY5Ym+odT6Qw=; b=CNqi09MtqItPkLTZQQTG2onfRxwIHyqy/8x0uQo+l+ecMkKQaG/7r08UVxyEHyf82l 2/4CYdO28aIrmLpFoLDE6E7WnsrRuW1fH6YjuWIUP5bOcAlCkb1alA804QW37MQaEyf0 UI4DODgzpNXjnlr2D6IjjPTqB1CmXkPFNTCJTg+NU9J9t0tBNpBmoFDNFPIPL+Ikxwgl PglWu+BU1ZHzRg4bQNU3i6bfsYGZfuugRmKJgseAPe0Xns+0tgIKXzfY1D5CSOmzUOSn Ho+wyIoyJcpg1ILDpkqeU7aXsNVwv08XEPmvi7K+OCflEfwg0GE0Hg7UD2i94fp/Zff2 yMCg== X-Gm-Message-State: AKS2vOzh+XBHdpzPG4y7NrY9GHlZkZRHdlUO0v48mkZKSAKtYX5BqG3L vc74UpnCsiHLgQ== X-Received: by 10.223.176.61 with SMTP id f58mr13231360wra.74.1498483629840; Mon, 26 Jun 2017 06:27:09 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id p34sm10079679wrc.66.2017.06.26.06.27.08 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Jun 2017 06:27:09 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> Date: Mon, 26 Jun 2017 15:27:07 +0200 Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> To: "Andrey V. Elsukov" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:27:12 -0000 > On 26 Jun 2017, at 15:13, Andrey V. Elsukov wrote: >=20 > I think it is not mlxen specific problem, we have the same symptoms = with > ixgbe(4) driver too. To avoid the problem we have patches that are > disable using of 9k mbufs, and instead only use 4k mbufs. Interesting feedback Andrey, thank you ! The problem may be then "general". So you still use large MTU (>=3D9000) but only allocating 4k mbufs, as a = workaround ?= From owner-freebsd-net@freebsd.org Mon Jun 26 13:28:19 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D736BD8715B; Mon, 26 Jun 2017 13:28:19 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward5j.cmail.yandex.net (forward5j.cmail.yandex.net [IPv6:2a02:6b8:0:1630::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7083C79618; Mon, 26 Jun 2017 13:28:19 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1m.mail.yandex.net (smtp1m.mail.yandex.net [77.88.61.132]) by forward5j.cmail.yandex.net (Yandex) with ESMTP id 5E92420EAC; Mon, 26 Jun 2017 16:28:08 +0300 (MSK) Received: from smtp1m.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp1m.mail.yandex.net (Yandex) with ESMTP id D32AA63C0F61; Mon, 26 Jun 2017 16:28:06 +0300 (MSK) Received: by smtp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id TNF1vv7VYM-S5X4233p; Mon, 26 Jun 2017 16:28:05 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1498483685; bh=YDJYvliDEulS1o4liFPL19zv/iJDJX1hqtF0z7hzcBA=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=KH8C6X7XlFLqX8TF3QKLlzQVbr1p0hTT5y6ZHFWqaJLBCwhNB5pWdlLdQKHaawmOB IIdtueBj+6DvneXkhtuT9bZiMQFC8ZiNWiL9ztALChnNWb2zB+3+IIdZPYuTx+s9PH yCI5N2LvrRusuR3FLB9OcL8R6T3xZkfmglhx3HuU= Authentication-Results: smtp1m.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0,1 0,1 0 Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: Date: Mon, 26 Jun 2017 16:25:28 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9jWKaCsF82R6ID2MvOwPFP64eGWl7m89A" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:28:19 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9jWKaCsF82R6ID2MvOwPFP64eGWl7m89A Content-Type: multipart/mixed; boundary="xqTNpa3hGbXomXjTul6pEVMVTJWshQCR8"; protected-headers="v1" From: "Andrey V. Elsukov" To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> In-Reply-To: <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> --xqTNpa3hGbXomXjTul6pEVMVTJWshQCR8 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 26.06.2017 16:27, Ben RUBSON wrote: >=20 >> On 26 Jun 2017, at 15:13, Andrey V. Elsukov wrote:= >> >> I think it is not mlxen specific problem, we have the same symptoms wi= th >> ixgbe(4) driver too. To avoid the problem we have patches that are >> disable using of 9k mbufs, and instead only use 4k mbufs. >=20 > Interesting feedback Andrey, thank you ! > The problem may be then "general". > So you still use large MTU (>=3D9000) but only allocating 4k mbufs, as = a workaround ? Yes. --=20 WBR, Andrey V. Elsukov --xqTNpa3hGbXomXjTul6pEVMVTJWshQCR8-- --9jWKaCsF82R6ID2MvOwPFP64eGWl7m89A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAllRC0gACgkQAcXqBBDI oXolKAf/VzLKbEtViQOs0S8iDBHHtg1+nTeDCtQCWdLq+dyLhwPWcDrHiSJ6xr1m Xvt4nuuXb1GFcLNBU/ewO5Kg2qM4qbnPN/k3OdeSdBiYfFKH1GwBVmwvDOkVhb0E gCoj4ambpXb18DElABEOIl/4xta91W4zpu+CGPPYG9QfsnMXwDh8xBaOtus/Ktar AKlUDo4rE9UaCDYuvVAa0bqSgHbgJ9B/xJKzGppNwDtFt/B364WByEbL5yDIvkR1 hrOVVi1qp7P0fLcCboD91elipEmZL4uDoVDZOFL/kj+yfnVuPtp2PqAkCCxyz3Rw UyJnnUZTBB4fDHhrIK4obakxpLgkQQ== =KQil -----END PGP SIGNATURE----- --9jWKaCsF82R6ID2MvOwPFP64eGWl7m89A-- From owner-freebsd-net@freebsd.org Mon Jun 26 13:29:51 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0137D8725B; Mon, 26 Jun 2017 13:29:51 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C1357972D; Mon, 26 Jun 2017 13:29:51 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22d.google.com with SMTP id 62so1366155wmw.1; Mon, 26 Jun 2017 06:29:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6rcpEB/MMsV7jfh73S50iuteAuxdvGYhwUPWXJNu33k=; b=eJN1C5yk9Fu2CBWogpToPkxhW5Sl48pHpqP6qHNQTN3qfN4TCWk6JEodUiWrxp79sE kL8VPp6TFij8yyZ9JZ833hTb7BW9aKpA98NcggQAsIXKbaZ8D/V+MeZGBiBG7OU47pjU iqXwt+hGXWMGWvsXVB8NPbWjHUN0ntqWnC+D4Xz/HfNam8Ba4Vu8fxg6AcCcANm4GgVz kQwRqOCIchs1GtiC8Ys/PA7RA2ropciPWplOH2Zei7Ch1GgGZH7i26jIpoJCPoabAT30 EWGoH7ADYbL4cJy//b+pjeOeA+YY1QU2yND87m8G9Y2ExrQi3kAfBSJr2I5P5/xxA3Oi Fbjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6rcpEB/MMsV7jfh73S50iuteAuxdvGYhwUPWXJNu33k=; b=A+eZ0gG+uNWQ77/hSN1MB2mjmjDxZOtdml6NHKN2lY6g9ECyoQ9ghtuU3By9y/PiDU aawnILIJneVcIyIcxlT2YN6vDnHqSQMgjbMScOYZvTFN7XIQR3T+3AFrCAMQpFQvY1aE Wpqnn8tf8CA9yQ4abheGRy7Z6RzPU8SobPSBXocJwQjNZtnWklE7GeMMmTdiiqNAa8S4 zGCYyoWTfx0BXvPtG9fnHtm9NH1HPG4xOxGMMM9LGhPHlK3tDSR+MiYUqnG5lmaQkYsm nlk1SYOEZHqulwZ7yLg8BA37F/HDjcij2JePiXgThul1AUN4UeAG81ZvCoHN24q5YImW LIfA== X-Gm-Message-State: AKS2vOy9aZQHgtnFMdDDDtyT6wXjYFUhw58uDh5TPiAmlqmVIgZjWFp5 fxEQ1uII/ryOPw== X-Received: by 10.28.136.4 with SMTP id k4mr117790wmd.4.1498483789545; Mon, 26 Jun 2017 06:29:49 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id x71sm109771wmd.32.2017.06.26.06.29.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Jun 2017 06:29:49 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: Date: Mon, 26 Jun 2017 15:29:48 +0200 Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> To: "Andrey V. Elsukov" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:29:51 -0000 > On 26 Jun 2017, at 15:25, Andrey V. Elsukov wrote: >=20 > On 26.06.2017 16:27, Ben RUBSON wrote: >>=20 >>> On 26 Jun 2017, at 15:13, Andrey V. Elsukov = wrote: >>>=20 >>> I think it is not mlxen specific problem, we have the same symptoms = with >>> ixgbe(4) driver too. To avoid the problem we have patches that are >>> disable using of 9k mbufs, and instead only use 4k mbufs. >>=20 >> Interesting feedback Andrey, thank you ! >> The problem may be then "general". >> So you still use large MTU (>=3D9000) but only allocating 4k mbufs, = as a workaround ? >=20 > Yes. Is it a kernel patch or a driver/ixgbe patch ?= From owner-freebsd-net@freebsd.org Mon Jun 26 13:39:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A671BD8782E; Mon, 26 Jun 2017 13:39:28 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward4h.cmail.yandex.net (forward4h.cmail.yandex.net [IPv6:2a02:6b8:0:f35::111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 438B279CF3; Mon, 26 Jun 2017 13:39:28 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp4o.mail.yandex.net (smtp4o.mail.yandex.net [37.140.190.29]) by forward4h.cmail.yandex.net (Yandex) with ESMTP id 191F620C8F; Mon, 26 Jun 2017 16:39:16 +0300 (MSK) Received: from smtp4o.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp4o.mail.yandex.net (Yandex) with ESMTP id 4E9B86C00CA3; Mon, 26 Jun 2017 16:39:13 +0300 (MSK) Received: by smtp4o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id hhLmE7UlhM-dCiKoYsj; Mon, 26 Jun 2017 16:39:13 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1498484353; bh=uno0HOfcwwJLZsdy+hh8yndMU0a7DYP/+WoPL12EUFY=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=IgN6CcT3wUKfDbZqWzp+XdS2UczF77h8cMSVnVmDvMHRzaIW3xVBwVSxI7CT7TjUw Pr8XHnN3bg+N2nr1mZQm8uReMx9yliFna0quXhLcDxbUa4/k01ub2AhVOisIuN5LbJ 6gmn8lIMqk9ICFg9X7F7KwqSVN6LhiMYNan6CNaU= Authentication-Results: smtp4o.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0,1 0,1 0 Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> Date: Mon, 26 Jun 2017 16:36:35 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="8VTMwHwj940cm1xRx4AhDLWSOS98RD7OQ" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:39:28 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --8VTMwHwj940cm1xRx4AhDLWSOS98RD7OQ Content-Type: multipart/mixed; boundary="g9A9E4Di6C80mHn9j0G28L5VKTWTg18l6"; protected-headers="v1" From: "Andrey V. Elsukov" To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Message-ID: <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> Subject: Re: mbuf_jumbo_9k & iSCSI failing References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> In-Reply-To: --g9A9E4Di6C80mHn9j0G28L5VKTWTg18l6 Content-Type: multipart/mixed; boundary="------------FF1D5726D14DBD643DE462AD" Content-Language: en-US This is a multi-part message in MIME format. --------------FF1D5726D14DBD643DE462AD Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 26.06.2017 16:29, Ben RUBSON wrote: >=20 >> On 26 Jun 2017, at 15:25, Andrey V. Elsukov wrote:= >> >> On 26.06.2017 16:27, Ben RUBSON wrote: >>> >>>> On 26 Jun 2017, at 15:13, Andrey V. Elsukov wrot= e: >>>> >>>> I think it is not mlxen specific problem, we have the same symptoms = with >>>> ixgbe(4) driver too. To avoid the problem we have patches that are >>>> disable using of 9k mbufs, and instead only use 4k mbufs. >>> >>> Interesting feedback Andrey, thank you ! >>> The problem may be then "general". >>> So you still use large MTU (>=3D9000) but only allocating 4k mbufs, a= s a workaround ? >> >> Yes. >=20 > Is it a kernel patch or a driver/ixgbe patch ? I attached it. --=20 WBR, Andrey V. Elsukov --------------FF1D5726D14DBD643DE462AD Content-Type: text/x-patch; name="0004-Add-m_preferredsize-and-use-it-in-all-intel-s-driver.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename*0="0004-Add-m_preferredsize-and-use-it-in-all-intel-s-driver.pa"; filename*1="tch" =46rom 57b4789b7f6699a581ded2f4e07c7b12737af1e7 Mon Sep 17 00:00:00 2001 From: "Andrey V. Elsukov" Date: Thu, 6 Oct 2016 14:56:37 +0300 Subject: [PATCH 04/65] Add m_preferredsize() and use it in all intel's drivers. --- sys/dev/e1000/if_em.c | 7 +------ sys/dev/e1000/if_igb.c | 7 +------ sys/dev/ixgbe/if_ix.c | 5 +---- sys/dev/ixgbe/if_ixv.c | 5 +---- sys/dev/ixl/if_ixlv.c | 5 +---- sys/dev/ixl/ixl_pf_main.c | 5 +---- sys/kern/kern_mbuf.c | 35 +++++++++++++++++++++++++++++++++++ sys/sys/mbuf.h | 1 + 8 files changed, 42 insertions(+), 28 deletions(-) diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c index 7e2690eae08..1af66b7c519 100644 --- a/sys/dev/e1000/if_em.c +++ b/sys/dev/e1000/if_em.c @@ -1421,12 +1421,7 @@ em_init_locked(struct adapter *adapter) ** Figure out the desired mbuf ** pool for doing jumbos */ - if (adapter->hw.mac.max_frame_size <=3D 2048) - adapter->rx_mbuf_sz =3D MCLBYTES; - else if (adapter->hw.mac.max_frame_size <=3D 4096) - adapter->rx_mbuf_sz =3D MJUMPAGESIZE; - else - adapter->rx_mbuf_sz =3D MJUM9BYTES; + adapter->rx_mbuf_sz =3D m_preferredsize(adapter->hw.mac.max_frame_size)= ; =20 /* Prepare receive descriptors and buffers */ if (em_setup_receive_structures(adapter)) { diff --git a/sys/dev/e1000/if_igb.c b/sys/dev/e1000/if_igb.c index 8e018995029..bfaecae1f71 100644 --- a/sys/dev/e1000/if_igb.c +++ b/sys/dev/e1000/if_igb.c @@ -1325,12 +1325,7 @@ igb_init_locked(struct adapter *adapter) ** Figure out the desired mbuf pool ** for doing jumbo/packetsplit */ - if (adapter->max_frame_size <=3D 2048) - adapter->rx_mbuf_sz =3D MCLBYTES; - else if (adapter->max_frame_size <=3D 4096) - adapter->rx_mbuf_sz =3D MJUMPAGESIZE; - else - adapter->rx_mbuf_sz =3D MJUM9BYTES; + adapter->rx_mbuf_sz =3D m_preferredsize(adapter->max_frame_size); =20 /* Prepare receive descriptors and buffers */ if (igb_setup_receive_structures(adapter)) { diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index cf2231dc8fc..26fce2704ba 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -1118,10 +1118,7 @@ ixgbe_init_locked(struct adapter *adapter) ixgbe_set_multi(adapter); =20 /* Determine the correct mbuf pool, based on frame size */ - if (adapter->max_frame_size <=3D MCLBYTES) - adapter->rx_mbuf_sz =3D MCLBYTES; - else - adapter->rx_mbuf_sz =3D MJUMPAGESIZE; + adapter->rx_mbuf_sz =3D m_preferredsize(adapter->max_frame_size); =20 /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { diff --git a/sys/dev/ixgbe/if_ixv.c b/sys/dev/ixgbe/if_ixv.c index 80fb1b34be3..5062affb779 100644 --- a/sys/dev/ixgbe/if_ixv.c +++ b/sys/dev/ixgbe/if_ixv.c @@ -698,10 +698,7 @@ ixv_init_locked(struct adapter *adapter) ** Determine the correct mbuf pool ** for doing jumbo/headersplit */ - if (ifp->if_mtu > ETHERMTU) - adapter->rx_mbuf_sz =3D MJUMPAGESIZE; - else - adapter->rx_mbuf_sz =3D MCLBYTES; + adapter->rx_mbuf_sz =3D m_preferredsize(ifp->if_mtu); =20 /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { diff --git a/sys/dev/ixl/if_ixlv.c b/sys/dev/ixl/if_ixlv.c index c447c34689e..608d784bfee 100644 --- a/sys/dev/ixl/if_ixlv.c +++ b/sys/dev/ixl/if_ixlv.c @@ -904,10 +904,7 @@ ixlv_init_locked(struct ixlv_sc *sc) =20 ixl_init_tx_ring(que); =20 - if (vsi->max_frame_size <=3D MCLBYTES) - rxr->mbuf_sz =3D MCLBYTES; - else - rxr->mbuf_sz =3D MJUMPAGESIZE; + rxr->mbuf_sz =3D m_preferredsize(vsi->max_frame_size); ixl_init_rx_ring(que); } =20 diff --git a/sys/dev/ixl/ixl_pf_main.c b/sys/dev/ixl/ixl_pf_main.c index d8da4cfee10..8600b0f931e 100644 --- a/sys/dev/ixl/ixl_pf_main.c +++ b/sys/dev/ixl/ixl_pf_main.c @@ -2067,10 +2067,7 @@ ixl_initialize_vsi(struct ixl_vsi *vsi) ixl_init_tx_ring(que); =20 /* Next setup the HMC RX Context */ - if (vsi->max_frame_size <=3D MCLBYTES) - rxr->mbuf_sz =3D MCLBYTES; - else - rxr->mbuf_sz =3D MJUMPAGESIZE; + rxr->mbuf_sz =3D m_preferredsize(vsi->max_frame_size); =20 u16 max_rxmax =3D rxr->mbuf_sz * hw->func_caps.rx_buf_chain_len; =20 diff --git a/sys/kern/kern_mbuf.c b/sys/kern/kern_mbuf.c index 0d0c1c86b16..7c10cedb075 100644 --- a/sys/kern/kern_mbuf.c +++ b/sys/kern/kern_mbuf.c @@ -103,6 +103,10 @@ int nmbjumbop; /* limits number of page size jumbo= clusters */ int nmbjumbo9; /* limits number of 9k jumbo clusters */ int nmbjumbo16; /* limits number of 16k jumbo clusters */ =20 +static int nojumbobuf; /* Use MCLBYTES mbufs */ +static int nojumbo9buf; /* Use either MCLBYTES or MJUMPAGESIZE */ +static int nojumbo16buf; /* Use any mbuf size less than MJUM16BYTES */ + static quad_t maxmbufmem; /* overall real memory limit for all mbufs */ =20 SYSCTL_QUAD(_kern_ipc, OID_AUTO, maxmbufmem, CTLFLAG_RDTUN | CTLFLAG_NOF= ETCH, &maxmbufmem, 0, @@ -151,6 +155,16 @@ tunable_mbinit(void *dummy) if (nmbufs < nmbclusters + nmbjumbop + nmbjumbo9 + nmbjumbo16) nmbufs =3D lmax(maxmbufmem / MSIZE / 5, nmbclusters + nmbjumbop + nmbjumbo9 + nmbjumbo16); + /* + * Defaults to disable 9/16-kbyte pages + */ + nojumbobuf =3D 0; + nojumbo9buf =3D 1; + nojumbo16buf =3D 1; + + TUNABLE_INT_FETCH("kern.ipc.nojumbobuf", &nojumbobuf); + TUNABLE_INT_FETCH("kern.ipc.nojumbo9buf", &nojumbo9buf); + TUNABLE_INT_FETCH("kern.ipc.nojumbo16buf", &nojumbo16buf); } SYSINIT(tunable_mbinit, SI_SUB_KMEM, SI_ORDER_MIDDLE, tunable_mbinit, NU= LL); =20 @@ -261,6 +275,27 @@ SYSCTL_PROC(_kern_ipc, OID_AUTO, nmbufs, CTLTYPE_INT= |CTLFLAG_RW, "Maximum number of mbufs allowed"); =20 /* + * Determine the correct mbuf pool + * for given mtu size + */ +int +m_preferredsize(int mtu) +{ + int size; + + if (mtu <=3D 2048 || nojumbobuf !=3D 0) + size =3D MCLBYTES; + else if (mtu <=3D 4096 || nojumbo9buf !=3D 0) + size =3D MJUMPAGESIZE; + else if (mtu <=3D 9216 || nojumbo16buf !=3D 0) + size =3D MJUM9BYTES; + else + size =3D MJUM16BYTES; + + return (size); +} + +/* * Zones from which we allocate. */ uma_zone_t zone_mbuf; diff --git a/sys/sys/mbuf.h b/sys/sys/mbuf.h index fdd9931515d..b6a81b05e3b 100644 --- a/sys/sys/mbuf.h +++ b/sys/sys/mbuf.h @@ -606,6 +606,7 @@ u_int m_length(struct mbuf *, struct mbuf **); int m_mbuftouio(struct uio *, struct mbuf *, int); void m_move_pkthdr(struct mbuf *, struct mbuf *); int m_pkthdr_init(struct mbuf *, int); +int m_preferredsize(int); struct mbuf *m_prepend(struct mbuf *, int, int); void m_print(const struct mbuf *, int); struct mbuf *m_pulldown(struct mbuf *, int, int, int *); --=20 2.12.1 --------------FF1D5726D14DBD643DE462AD-- --g9A9E4Di6C80mHn9j0G28L5VKTWTg18l6-- --8VTMwHwj940cm1xRx4AhDLWSOS98RD7OQ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAllRDeMACgkQAcXqBBDI oXqqvAf6AwhLtFohbNT9kfkP6P0pMIGoXCYwQ7ACxYVLmTLzKorGGV4aj+DalTjv Dv7H2ICYXBcESgH8xjZgMKeAVxIfMlsvGVbRwQs3rnSO9bMjGLXsPxcD6ymvZf4L tgxm4aBPyPFevdBD6DdU7bdfv+Ml1c15iQ/Vr5khQaplMkcw2q0mUI6efJD6agp0 5fjd7kxrHDxranQr+DcW6lw+pd4GQakBLy5JNODTESMOc4DaUbhGVE79nRvMqTMU LjJAWxQGqvwRvvy3RkWuCczjuBelM0Cb9U8HxPbDzumyXbY113raLidGJbabnuul YPGe76RwAwFQiFkVaLiplL8pMkaAZw== =0JjL -----END PGP SIGNATURE----- --8VTMwHwj940cm1xRx4AhDLWSOS98RD7OQ-- From owner-freebsd-net@freebsd.org Mon Jun 26 13:57:51 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6E12D87ED2; Mon, 26 Jun 2017 13:57:51 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x242.google.com (mail-wr0-x242.google.com [IPv6:2a00:1450:400c:c0c::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 520A17A71D; Mon, 26 Jun 2017 13:57:51 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x242.google.com with SMTP id 77so29375157wrb.3; Mon, 26 Jun 2017 06:57:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=BEEoNVE2CrKbx9gHAYaoAeYCLOeLKlHzIeLaw57IxhM=; b=UsYDHf2MKfhWumAmRebPCb6bCSggmRSDFnFLjuvilFmech2TywdD/ndov24EjhoymJ Z2Xx97kkP6A+660RgmcnK6bOF/Lf6JfFns6Ja7iq+72PLpdf6/s0hb2M+xfFpMy5xq06 p/SSkkuQFuwGoOEBFurEmKk2FpCHmmPH6oT9iovIDwColSEWQftxecL3BcSmutekcZzw MlCIxloPgnmHv6qCQBeiHNyUybf0uA1QzEu/DJXmu3gwiLMSRHHAfEg59psgYMWytE7E aTRhzAte7rFHY6zG+L+RI3FWJ/uA+Z/Ow2aF4+CNyMhFWel2kRhCSLA9q4AU6JfSR0Mk KX/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=BEEoNVE2CrKbx9gHAYaoAeYCLOeLKlHzIeLaw57IxhM=; b=EPOM7iZ+mG1ijkPCVmnie3cdGRmNCjdIDB7/RhaLEC7EPf+Ft9cCAYb42/Q/hV7nOu 6KSmTAcu210B1VyQoWEilui8BNa1ZWvvDl1z7u9AW9pSHOoPQn8/Q7oWA5mH3n7hsdaW IyygC2ElKuo+wtVMSbBAa3jX6pIbaa67C4o3yDbx+lL9q1qeCQ3NhJ9VRQlcdTvtGCxR nYGGK9fwdUGOJ+io9s9a/4llFsUHUizmN3bdptazJWCiePFDqSwwrmjb6iEP+kvJMZQD kgH5MPTFPNgyRzyhaoxtpwMShGkXNLXbrrSwHOfpcAjGHEZ+nTa/o99GDKSM2YXuJAtQ lAgw== X-Gm-Message-State: AKS2vOxJrNXYjufhTjxToBVUxgc+b5YxWwlOfkrsJs+hdAs3CzlYj8wc Cuxfkljmx/0vdw== X-Received: by 10.223.151.51 with SMTP id r48mr14803383wrb.189.1498485469753; Mon, 26 Jun 2017 06:57:49 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id p87sm209778wma.2.2017.06.26.06.57.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Jun 2017 06:57:49 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> Date: Mon, 26 Jun 2017 15:57:48 +0200 Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <2C291A70-B6DD-4E21-9106-4FE023E9EAFE@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> To: "Andrey V. Elsukov" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 13:57:51 -0000 > On 26 Jun 2017, at 15:36, Andrey V. Elsukov wrote: >=20 > On 26.06.2017 16:29, Ben RUBSON wrote: >>=20 >>> On 26 Jun 2017, at 15:25, Andrey V. Elsukov = wrote: >>>=20 >>> On 26.06.2017 16:27, Ben RUBSON wrote: >>>>=20 >>>>> On 26 Jun 2017, at 15:13, Andrey V. Elsukov = wrote: >>>>>=20 >>>>> I think it is not mlxen specific problem, we have the same = symptoms with >>>>> ixgbe(4) driver too. To avoid the problem we have patches that are >>>>> disable using of 9k mbufs, and instead only use 4k mbufs. >>>>=20 >>>> Interesting feedback Andrey, thank you ! >>>> The problem may be then "general". >>>> So you still use large MTU (>=3D9000) but only allocating 4k mbufs, = as a workaround ? >>>=20 >>> Yes. >>=20 >> Is it a kernel patch or a driver/ixgbe patch ? >=20 > I attached it. Thank you ! The idea of new sysctls to enable/disable the workaround is nice. Should be easy to modify to use with mlx4_en, waiting for Ryan specific = work on this driver.=20 I found a similar issue, reported date : 2013-10-28 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183381 FreeBSD certainly needs a general solid patch != From owner-freebsd-net@freebsd.org Mon Jun 26 14:00:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24B0ED87FCB for ; Mon, 26 Jun 2017 14:00:54 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C41C87A892 for ; Mon, 26 Jun 2017 14:00:53 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1498484698-0a7b8d16e45f9920001-QdxwpM Received: from mordor.lan (77.109.96.171.adsl.dyn.edpnet.net [77.109.96.171]) by relay-b02.edpnet.be with ESMTP id iQJ7RsBX2SeU6S2e (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 26 Jun 2017 15:44:59 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 77.109.96.171.adsl.dyn.edpnet.net[77.109.96.171] X-Barracuda-Apparent-Source-IP: 77.109.96.171 Date: Mon, 26 Jun 2017 15:44:58 +0200 From: Julien Cigar To: "Andrey V. Elsukov" Cc: Ryan Stone , Ben RUBSON , FreeBSD Net , "freebsd-scsi@freebsd.org" Subject: Re: mbuf_jumbo_9k & iSCSI failing Message-ID: <20170626134458.GT43966@mordor.lan> X-ASG-Orig-Subj: Re: mbuf_jumbo_9k & iSCSI failing References: <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="TgCXP+xznsSrEyty" Content-Disposition: inline In-Reply-To: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> User-Agent: Mutt/1.8.2 (2017-04-18) X-Barracuda-Connect: 77.109.96.171.adsl.dyn.edpnet.net[77.109.96.171] X-Barracuda-Start-Time: 1498484698 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-Scan-Msg-Size: 1814 X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4999 1.0000 0.0000 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=6.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.40248 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 14:00:54 -0000 --TgCXP+xznsSrEyty Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 26, 2017 at 04:13:33PM +0300, Andrey V. Elsukov wrote: > On 25.06.2017 18:32, Ryan Stone wrote: > > Having looking at the original email more closely, I see that you showe= d an > > mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > > clusters increase while you are far below the zone's limit means that > > you're definitely running into the bug I'm describing, and this bug cou= ld > > plausibly cause the iSCSI errors that you describe. > >=20 > > The issue is that the newer version of the driver tries to allocate a > > single buffer to accommodate an MTU-sized packet. Over time, however, > > memory will become fragmented and eventually it can become impossible to > > allocate a 9k physically contiguous buffer. When this happens the driv= er > > is unable to allocate buffers to receive packets and is forced to drop > > them. Presumably, if iSCSI suffers too many packet drops it will termi= nate > > the connection. The older version of the driver limited itself to > > page-sized buffers, so it was immune to issues with memory fragmentatio= n. >=20 > I think it is not mlxen specific problem, we have the same symptoms with > ixgbe(4) driver too. To avoid the problem we have patches that are > disable using of 9k mbufs, and instead only use 4k mbufs. I had the same issue on a lightly loaded HP DL20 machine (BCM5720=20 chipsets), 8GB of RAM, running 10.3. Problem usually happens within 30 days with 9k jumbo clusters allocation failure. >=20 > --=20 > WBR, Andrey V. Elsukov >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --TgCXP+xznsSrEyty Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE7vn2l0to0nV7EWolsrs3EKIEI8AFAllRD9cACgkQsrs3EKIE I8Dj9w//XKLEOkjmTdf9HKiS5Dhe7nLJyFP5RFVXqSU4gx9b6oxh9jHKbBU+10iK v+yxFye/LtQKDabaOvwiGhMhhcYt2zVLWAKecpPwSxFE4KisW1KUw+PkzMXVoEP9 d9MXC3iPbTG3vNBFqZtK+VaXNQc3BZ6ZhgGMjO85Mbn2bp841kEtJROfPLvpYPfF yDcCZOAvD/ALzTprxRWzFZbRCl7TLJRUFCLHIGScm4B/QXbzdb/uqw9U265DQneO cDvig+wddfrC8DI6nhUhkv/o6CvN4pqIm66UZRCGyfni12MnMfXPyKP+ohrZPPGR btrUAzb0lBBM6E1Vmpi37IHERKR22wRsUkB52//ffJwHmmziR8ytM6Rns3V9xgkJ Qf0+PidI5fsqltf47IM47iXgwT04+FSWvZ+aUOv67nRPttAQdoZehXodU/ECnb22 jRMTIISu6p9Jo6ihwoMqFXGqJdOHAumLLu4uzXpVTcgSa6Qk6ei868PxBLo5EoEJ o2MXuz97EnlAfhfpo9zE0uzHnqao8QOrdQgz8CEBv3+0lozuTYGO5hgx+4O8dpl4 531pADsW6zlqTbCfTNJAEZeaGGG/fI8YmosQ4G1zdXL793O21QOwlkk7KXV9HX8U 9M179eg5NssUB2tBTXXqYAdo8iZf/c3staTo5DWj8qEtn78GIVk= =erJV -----END PGP SIGNATURE----- --TgCXP+xznsSrEyty-- From owner-freebsd-net@freebsd.org Mon Jun 26 15:11:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5667D89845; Mon, 26 Jun 2017 15:11:02 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 653417CA2D; Mon, 26 Jun 2017 15:11:02 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id b184so435902wme.1; Mon, 26 Jun 2017 08:11:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=2Ivhmt0ZiFxkr4AXYgDELGiGv49zFBsBaKqq+F1OS/g=; b=jjHwTPfeZ7TadFZLAtQiax+48/4+0QTOMje597q/oH2bBVr8dZ9Y1lwvOr+2N1WefQ vKEqxqv+gmT/GDZ8jGOXr+derv5yRRqtHHmqdTVdz0DRLcXLCIiiJixXXdQ3Ex+78gnE XiuvrhyWjURYtpV8HCY0eZarDG53bGiVzELYVZ1DoPB5wkQCxxPKlhnwU07u5LPBUPkA qs13S2pupiYP0dankaTs2KqVLzQgD0RiJLAEcy+zLHQ+s2UCSszwlGla9oFrNKGqOdFj EMvvi1WAYCVje3vx7VoVnk537pwOyOG1j+v2yWS4Ibxo9gH4YauT/6/GcrnpoKVdWnEO peGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=2Ivhmt0ZiFxkr4AXYgDELGiGv49zFBsBaKqq+F1OS/g=; b=t9OZCavu5QPsdclyvK94k/+idtKcJilPvGcxREAIYoErLMmh6lDt3spkoEH3Gbkchk QZBokODijZl9MoOi0sPNrVcQtasWZfsNsQaeCVikpVsYBK6VsaGJXLBIDNm3KvPiTMB+ MfefBCCyolgOC9WTMdwBwD+/X9jps3/B+PBQpAZVvZ5puj2I8bDyzNeRiFX+/0XXk55u 0cMbctWgAIkbIoJF2t0X5PFfgPc+PBvIclZiCQGnGe5DF77Lf5R71XjzYyYUBJbfJEM/ BAXdXM49BXNBVvQwB8eFdcRvgKClnvYz9JqOPXu0Sik6qZvN0vboJPJZw+MLo+ht/M35 nsFw== X-Gm-Message-State: AKS2vOwaAQveGH/nSTTkcTCDGz1EsKk/mUKZuZWfJIufjXi9OrDEroDj Kv/n/IFZWUNQTw== X-Received: by 10.28.130.196 with SMTP id e187mr56813wmd.24.1498489860792; Mon, 26 Jun 2017 08:11:00 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id m26sm19542171wrm.4.2017.06.26.08.10.59 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Jun 2017 08:11:00 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> Date: Mon, 26 Jun 2017 17:10:59 +0200 Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: 7bit Message-Id: <8CBA6288-BEB4-4301-8DAE-058B2348F909@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> To: "Andrey V. Elsukov" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 15:11:02 -0000 > On 26 Jun 2017, at 15:13, Andrey V. Elsukov wrote: > > I think it is not mlxen specific problem, we have the same symptoms with > ixgbe(4) driver too. To avoid the problem we have patches that are > disable using of 9k mbufs, and instead only use 4k mbufs. Another workaround is to decrease the MTU until 9K mbufs are not more used. On my systems it gives a 4072 bytes MTU. It solved the issue without having to reboot. Of course it's just a workaround, as decreasing MTU increases overhead... From owner-freebsd-net@freebsd.org Mon Jun 26 15:39:55 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C211D8A21B; Mon, 26 Jun 2017 15:39:55 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pg0-x234.google.com (mail-pg0-x234.google.com [IPv6:2607:f8b0:400e:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C82157D9B8; Mon, 26 Jun 2017 15:39:54 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by mail-pg0-x234.google.com with SMTP id f127so2076525pgc.0; Mon, 26 Jun 2017 08:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-disposition:in-reply-to:user-agent; bh=zemzJsAmUs18+RZbRidECL9wvjOpoixw5Ky1RCz89tE=; b=X6KFe6WSmYuBIwEyrcxtZozacEGGagfWkG88xHGqPng0Cwc1kB/muLWxJkJeZ05UjS MXvmPS4qL/ZdwG2zvnytEX7NV5uCOLKrpvE7RUIKHWrRXbkYXijbR0EXbbd+9wYj/RJZ J3tu7QU7zniSBImIx9XhtbQQJhdEBc4eg5uzWyrwOtJ13UiBdLzkqsE2zuLF5Kz7Q057 A2ZEw+Or2d31ML1xOw2xr9Y+9SgBTiZV8kcIdkN0XRlRxADOXtbNNlxBjhh1atOBzIR+ hVI4OggtK2qQG+Hv5YCrHFZCHKXrhOPdrmSxABcHtqHqeIqZ1QXave59/lrYf/U1ZE92 J8/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:reply-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=zemzJsAmUs18+RZbRidECL9wvjOpoixw5Ky1RCz89tE=; b=sjWGiS+vNyvvMqiuKRzF7993mf73OyJZuWScxMWGQ/P8peltJozIEuqdf2ndBdHbb5 Yr0Hn5Uuw+BuIEblcVaxnjtBYOZ2nhgR6R36kOSknVi0T6ag61vOKYLyLmN9WdK6blV0 FBS4n6n5UoDyWeRYpQRS+aFvrmRIwA4lg3Uhw4yz772S+41EcFAsDMHlKUoZxz+VpMpp +5H8zqT3rQQfBhyWdGMilxGi7PBmpvokaMFs//BPZibKpjqvXI2LmPWR1X+5ko2MyaXO nbRfH1aWeTmaVHMNR+Jb5fCxvV4BTsWRMondcIN68JgGf6deFKv6Lm4bhghleZzAFY4G jDhw== X-Gm-Message-State: AKS2vOxXnA6lcQy+SCnoayLiOqKtSf/SeRwM+sSXV/tUDYda98XJYlBn xYOy8ofdkPXOPg== X-Received: by 10.84.128.69 with SMTP id 63mr835077pla.54.1498491594314; Mon, 26 Jun 2017 08:39:54 -0700 (PDT) Received: from localhost ([1.227.152.47]) by smtp.gmail.com with ESMTPSA id n2sm709617pgd.26.2017.06.26.08.39.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Jun 2017 08:39:53 -0700 (PDT) From: YongHyeon PYUN X-Google-Original-From: "YongHyeon PYUN" Received: by localhost (sSMTP sendmail emulation); Tue, 27 Jun 2017 00:40:10 +0900 Date: Tue, 27 Jun 2017 00:40:10 +0900 To: Julien Cigar Cc: "Andrey V. Elsukov" , FreeBSD Net , Ryan Stone , Ben RUBSON , "freebsd-scsi@freebsd.org" Subject: Re: mbuf_jumbo_9k & iSCSI failing Message-ID: <20170626154010.GA2488@michelle.fasterthan.co.kr> Reply-To: pyunyh@gmail.com References: <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <20170626134458.GT43966@mordor.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170626134458.GT43966@mordor.lan> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 15:39:55 -0000 On Mon, Jun 26, 2017 at 03:44:58PM +0200, Julien Cigar wrote: > On Mon, Jun 26, 2017 at 04:13:33PM +0300, Andrey V. Elsukov wrote: > > On 25.06.2017 18:32, Ryan Stone wrote: > > > Having looking at the original email more closely, I see that you showed an > > > mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > > > clusters increase while you are far below the zone's limit means that > > > you're definitely running into the bug I'm describing, and this bug could > > > plausibly cause the iSCSI errors that you describe. > > > > > > The issue is that the newer version of the driver tries to allocate a > > > single buffer to accommodate an MTU-sized packet. Over time, however, > > > memory will become fragmented and eventually it can become impossible to > > > allocate a 9k physically contiguous buffer. When this happens the driver > > > is unable to allocate buffers to receive packets and is forced to drop > > > them. Presumably, if iSCSI suffers too many packet drops it will terminate > > > the connection. The older version of the driver limited itself to > > > page-sized buffers, so it was immune to issues with memory fragmentation. > > > > I think it is not mlxen specific problem, we have the same symptoms with > > ixgbe(4) driver too. To avoid the problem we have patches that are > > disable using of 9k mbufs, and instead only use 4k mbufs. > > I had the same issue on a lightly loaded HP DL20 machine (BCM5720 > chipsets), 8GB of RAM, running 10.3. Problem usually happens > within 30 days with 9k jumbo clusters allocation failure. > This looks strange to me. If I recall correctly bge(4) does not request physically contiguous 9k jumbo buffers for BCM5720 so it wouldn't suffer from memory fragmentation. (It uses m_cljget() and takes advantage of extended RX BDs to handle up to 4 DMA segments). If your controller is either BCM5714/BCM5715 or BCM5780, it requires physically contiguous 9k jumbo buffers to handle jumbo frames though. From owner-freebsd-net@freebsd.org Mon Jun 26 16:26:22 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D2BDD8B61C; Mon, 26 Jun 2017 16:26:22 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-wr0-x229.google.com (mail-wr0-x229.google.com [IPv6:2a00:1450:400c:c0c::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9BCB7F36C; Mon, 26 Jun 2017 16:26:21 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-wr0-x229.google.com with SMTP id 77so147270960wrb.1; Mon, 26 Jun 2017 09:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=L4JxaB/ZQ4GTAlGrgHDF4xJYCRU6c3BJAy3OSKIVAZg=; b=HtsofQLi2EZWMxV5XTjidUCq+khQLoNxqvR/s5+Rd2eDv52OwpTcIiAXIAGQJaZFfj RB+2a62ZmjlO+dhC6xnX0kVI39uhc+pY8xb96rhtCjdeKgF+NpKt/IIjDfxmWczY9O8N t1NcoEs0VEAjhu9+SlHJ6ma7WUjYVL6ctQTcZBAg/EMWYkcdKPA7lmKPuOUQWt3iKomQ JuqQSO8+9CwFLEw4FofSO3fQQiMb9hAdNa/tv7r8QDP9ZEQ4FpSrGhejQlmFDKBlZQeY FNoxUxT9TR+AlXlxNsJGSuBggDRN52rgw210xu4qIYRqDRZnFldsc1nrLsy6fUdjxjg9 EeaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=L4JxaB/ZQ4GTAlGrgHDF4xJYCRU6c3BJAy3OSKIVAZg=; b=gyV+bTRhn3vCMkmxx1SFnx/IsJ/gJTOs+xxoRrCeYi+PFc7B4qhj5qZoUgQoRWhZn6 VG1Ol0yuabz/8wLCRCVneiNFvzK01sVPTT669YeGnw7YLQOiyfeMfaX2LpMdOFOdZP2F gw099Sf84FrsvJFw2qjUZrhrujOjkBTMWMqeTV5AMemfLqA90hwpLn3QlcZoApfP8QKE Fg/VZf6lF5aysk+9611VZzdDKlCM0Ad4/2Gjhg1jcpeVP5Telhy3Fi7vI+svDy93TNlC 3M2UR2NtoysE7ErFDv3K6i5HzztHIzFu3At8GEri/H/lG3e1V7vEOcB+wcNuMQJWM7w/ smlQ== X-Gm-Message-State: AKS2vOwqFFnI8Rr6f8H9ANmFvje4axFtFj9Oe5rj3kcvExJwhuQpbEkQ jv+D26Hpu8kg6o4wa3LargKofWqLDA== X-Received: by 10.223.160.40 with SMTP id k37mr13376113wrk.91.1498494380248; Mon, 26 Jun 2017 09:26:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.160.42 with HTTP; Mon, 26 Jun 2017 09:26:19 -0700 (PDT) In-Reply-To: <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> From: Matt Joras Date: Mon, 26 Jun 2017 09:26:19 -0700 Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: "Andrey V. Elsukov" Cc: Ben RUBSON , FreeBSD Net , Ryan Stone , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 16:26:22 -0000 On Mon, Jun 26, 2017 at 6:36 AM, Andrey V. Elsukov wrote: > On 26.06.2017 16:29, Ben RUBSON wrote: >> >>> On 26 Jun 2017, at 15:25, Andrey V. Elsukov wrote: >>> >>> On 26.06.2017 16:27, Ben RUBSON wrote: >>>> >>>>> On 26 Jun 2017, at 15:13, Andrey V. Elsukov wrote: >>>>> >>>>> I think it is not mlxen specific problem, we have the same symptoms with >>>>> ixgbe(4) driver too. To avoid the problem we have patches that are >>>>> disable using of 9k mbufs, and instead only use 4k mbufs. >>>> >>>> Interesting feedback Andrey, thank you ! >>>> The problem may be then "general". >>>> So you still use large MTU (>=9000) but only allocating 4k mbufs, as a workaround ? >>> >>> Yes. >> >> Is it a kernel patch or a driver/ixgbe patch ? > > I attached it. > > -- > WBR, Andrey V. Elsukov I didn't think that ixgbe(4) still suffered from this problem, and we use it in the same situations rstone mentioned above. Indeed, ixgbe(4) doesn't presently suffer from this problem (you can see that in your patch, as it is only effectively changing the other drivers), though it used to. It looks like it was first fixed to not to in r280182. From owner-freebsd-net@freebsd.org Mon Jun 26 16:33:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 52531D8B9C3; Mon, 26 Jun 2017 16:33:04 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward3j.cmail.yandex.net (forward3j.cmail.yandex.net [IPv6:2a02:6b8:0:1630::16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB2E67F964; Mon, 26 Jun 2017 16:33:03 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1j.mail.yandex.net (smtp1j.mail.yandex.net [95.108.130.59]) by forward3j.cmail.yandex.net (Yandex) with ESMTP id 19E3D20DAE; Mon, 26 Jun 2017 19:33:00 +0300 (MSK) Received: from smtp1j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp1j.mail.yandex.net (Yandex) with ESMTP id 47C8D3C80F56; Mon, 26 Jun 2017 19:32:56 +0300 (MSK) Received: by smtp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id AJ1ygWDziH-WuoiBls4; Mon, 26 Jun 2017 19:32:56 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1498494776; bh=+Nkw5KkQClkY3jfyBfJ8TMZbKDMZn/Ap1JQPuiUgof0=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=OOlT2o9BXjq0Bv4mtClYdLxHN0hLJ/LIPB1B3P4JQKGQ0M+8Vk/FRpvr+OEZo7RjV HfTF8wCe2bi0MjLU6M6C6UzQQ1Er96sxe7B1sJeKpRrxm8A6CU2a2FueRqpmcuB5KP Gogqsr7z04ah5SMW5cn6lR8PDJGyGQCYVKBcdQsI= Authentication-Results: smtp1j.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0,1 0,1 0,1 0 Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Matt Joras Cc: Ben RUBSON , FreeBSD Net , Ryan Stone , "freebsd-scsi@freebsd.org" References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: Date: Mon, 26 Jun 2017 19:30:22 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="LSl5TAswqMcmo5tKJaEuQNEMULCLPXSuI" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 16:33:04 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --LSl5TAswqMcmo5tKJaEuQNEMULCLPXSuI Content-Type: multipart/mixed; boundary="W5Mi1AjedTkUXLfJKKc2QlKE0GnXDnkwL"; protected-headers="v1" From: "Andrey V. Elsukov" To: Matt Joras Cc: Ben RUBSON , FreeBSD Net , Ryan Stone , "freebsd-scsi@freebsd.org" Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> <86D76532-92F4-479C-A714-126D007AD91F@gmail.com> <61f98b7d-f55d-aa0f-4aef-1bdfbc7086ff@yandex.ru> In-Reply-To: --W5Mi1AjedTkUXLfJKKc2QlKE0GnXDnkwL Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 26.06.2017 19:26, Matt Joras wrote: > I didn't think that ixgbe(4) still suffered from this problem, and we > use it in the same situations rstone mentioned above. Indeed, ixgbe(4) > doesn't presently suffer from this problem (you can see that in your > patch, as it is only effectively changing the other drivers), though > it used to. It looks like it was first fixed to not to in r280182. >=20 Yes, actually we have this patch since 8.x. Recent drivers aren't affected by this problem. iflib also has the code: #ifndef CONTIGMALLOC_WORKS else fl->ifl_buf_size =3D MJUMPAGESIZE; #else else if (sctx->isc_max_frame_size <=3D 4096) fl->ifl_buf_size =3D MJUMPAGESIZE; else if (sctx->isc_max_frame_size <=3D 9216) fl->ifl_buf_size =3D MJUM9BYTES; else fl->ifl_buf_size =3D MJUM16BYTES; #endif that seems by default doesn't use 9-16k mbufs. --=20 WBR, Andrey V. Elsukov --W5Mi1AjedTkUXLfJKKc2QlKE0GnXDnkwL-- --LSl5TAswqMcmo5tKJaEuQNEMULCLPXSuI Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAllRNp4ACgkQAcXqBBDI oXqBbwgAq7tK/6KFBl04+UzquamnCs4v85dAx65EG8gHFVAXkOSYW9rXBSieX2wU 9JPZNQmDF9eO6xv4oFHQg87bwIs6WEWKc3TO1iR+7mDycRDi/7dEEzmyi1Px4HFx 8gAnaF6VqTjixRfPRuXQ8eZXR6mKFGSVdiHwFrqZ6M6DTEZiqCxjAa7ZfF6mFSwH cs44QmzYCGP+bI6PIwF4ylI7gVgD7yWg/3zWxO0J5i3T+65+ZKAd4gznb09HxzHB R7mmoYWOsm/V9g07MlLhHkRzD9+Ozhm/dJk8F1WgP6gXvxh7etJHGuY9W7xl5Ic9 9MpgQB9xUoKrtqWPSHAX+pbx3tP94w== =Al7R -----END PGP SIGNATURE----- --LSl5TAswqMcmo5tKJaEuQNEMULCLPXSuI-- From owner-freebsd-net@freebsd.org Mon Jun 26 20:17:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A74C1D909A7 for ; Mon, 26 Jun 2017 20:17:04 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4062D3E7F for ; Mon, 26 Jun 2017 20:17:04 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id b184so7796342wme.1 for ; Mon, 26 Jun 2017 13:17:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=kCIixj2ovWBwkL9MmSIhc0QM4ggwVVYQUtZEucG6Up4=; b=Pw+9V+kq3ThS59SV1plS/aG1j73bqK9IkfdEcBOQsLspS+WKeaEypmPUVdwMIOxwri 5/N/auT410f5CEg4NTwEvHEzoukaSsfxKCxecc5rpTNOYsnLNgA6QPkUC7U3gd1nFwkb ZvGjr6wEn4EGtRGMlBIQ2frjuS/MXoKlobmvQMrmZXFdl4pi7y069/ynmJFaj+Ccf90L NcsS5T/SVwb8P7is7P8t6xNVOzWNGH/NwsqIjtVZqVr0A9rDxM6ceTC7aq0IYCAze32y tZQn+yceUkcnNzh/rwBGnAM2F0YEWx2aANw/m/5r+YRqSnKscHNWBDP62rn5SD43B1j8 eWjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=kCIixj2ovWBwkL9MmSIhc0QM4ggwVVYQUtZEucG6Up4=; b=eEj+rGd0ldlB/FfPw4H1ftoO3JqkX/CKXR1kr6JiLGS5Zs4W7pErGQ1z2WgwxgZoYt BBATKRSpIIl26PYxT6SKqDNcwM1iqxXnC3zeBuRtk/TljFjQ52MLc7y9R4MnPCrGd1Ce 2LW/m3HZ0NKWL/nyg5LpR4bO5b48MQs1mDbXnoYxOEU+rviCnlNq+6HJWgN+YQArbfAG VItzmoQ/y+wsgpF40YaucKUUxg5ySrHqtdEjnf0Oq6LaeIjRGw5dP/jx+NyLO0cSrjkW 3NgLbnSAT5PcwUv8MgN0iZm0Sn6RZXavUU1vDA2h79w/gkGXLZh8Y6KGtxn7mn2XNkxy o3pQ== X-Gm-Message-State: AKS2vOy/SXMTY2HgsX2TPtq0x0RcmryQupwcLObrAjV246nNiDLXmGWy tiYBKZ/tGuZ6fihdI443hOGfJGLPmFyJ X-Received: by 10.28.113.21 with SMTP id m21mr928597wmc.80.1498508222626; Mon, 26 Jun 2017 13:17:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.160.42 with HTTP; Mon, 26 Jun 2017 13:17:02 -0700 (PDT) From: Matt Joras Date: Mon, 26 Jun 2017 13:17:02 -0700 Message-ID: Subject: Request for reviewers for vlan(4) locking improvements To: FreeBSD Net Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 20:17:04 -0000 Hello, I am looking for people to give feedback on a review I've opened to improve the locking in vlan(4). Anyone who's done a fair amount of destroying vlan interfaces on live systems has probably run into panics in if_vlan. This is because there is no real synchronization to prevent a vlan interface from being destroyed while there are mbufs in the network going through its functions. Isilon's customers have hit panics like this, so I've reworked the locking to make destroying vlans safe on live systems, and fixed every instance of unsafe access I could find. If anyone has an interest in this work please review the revision: https://reviews.freebsd.org/D11370 Thanks, Matt Joras From owner-freebsd-net@freebsd.org Mon Jun 26 20:35:21 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B134D91710 for ; Mon, 26 Jun 2017 20:35:21 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg0-x22d.google.com (mail-pg0-x22d.google.com [IPv6:2607:f8b0:400e:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E09176556A for ; Mon, 26 Jun 2017 20:35:20 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg0-x22d.google.com with SMTP id u62so5252143pgb.3 for ; Mon, 26 Jun 2017 13:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=MBrkhB/nQ4rMVjBt8Psf127iUBJp70pGWThfI2v+Dxg=; b=loqTvy6iYvbq0xbCkZNWNRds8HIYVp5lCARoN3fWHC9DQcdqHgh/LYGVahRW+2Nwxv lXyGyoOW18YNDmDf6JDmoF4BtLRP/yU7AT5QVn8As31IzunolhGBBoZEJct7coXQeWUG 8LzQx3NgxsKdTVmkb9DjmIPE5FsLrcG4oDbsUSVvvUTiAKrygbkhM14R7BCNCETCRY5v KbZcj/KSEIFzPM5nlILhMuX2c1F0FP1ZlmX59t1y3SgOwxYI6l9nYa2lHzWAFUsdYH3o MOshueILLvextZ/aN9P9eG9Mq9qpQqPiAh6suS5xSDYQeFz/jwYHXu626fR5bF9NxyV1 AB7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=MBrkhB/nQ4rMVjBt8Psf127iUBJp70pGWThfI2v+Dxg=; b=bc85KWoeUoW/FsVLIQrkHcGIVIL0rGRWoqJeXTKixiRxtVqbddZFobCMkoYkUBinv7 oBIqCpvJ8TiTLW8HpilIrIutAyqF25rNNkXIkiqIas+JYscmc4izjYYAalKzbtpa94R5 42TVIrud6IljE6HrUPyt5Fll/jhZ9FBBvTM9bvLIGyahUF4XGN3fzI9tDcBnY0fQXuBv SfAdDhtC9VeYjyXQDJdfvhUiobwlLzbbDBGqXHJ6cu01QCG0cZHB5l7zDepAdh8nFHyJ CxvGnGlIAGx/BLZB5/f5xVo62EvXY3ANUhQqyXK5ZzoARLSmcPCjsmJZDe7O6NAtHhMK au3A== X-Gm-Message-State: AKS2vOyutVLHpsVTe0/nKMVewrfe7y1KPRZ2lFc+1Ol8FSq63b9n4ygH UA4+C4TuGjzN5otkkmgz1ZuN+LEj/Q== X-Received: by 10.84.171.193 with SMTP id l59mr2042574plb.139.1498509319998; Mon, 26 Jun 2017 13:35:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.135.80 with HTTP; Mon, 26 Jun 2017 13:35:19 -0700 (PDT) In-Reply-To: <84CB0795-B28E-46DF-9593-4C1BAAB7DDF5@pasteur.fr> References: <84CB0795-B28E-46DF-9593-4C1BAAB7DDF5@pasteur.fr> From: Navdeep Parhar Date: Mon, 26 Jun 2017 13:35:19 -0700 Message-ID: Subject: Re: Sporadic TCP/RST sent to client To: Youssef GHORBAL Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 20:35:21 -0000 On Thu, Jun 22, 2017 at 3:57 PM, Youssef GHORBAL wrote: > Hello, > > I'm having an issue with a FreeBSD 11 based system, sending sporadically TCP/RST to clients after initial TCP session correctly initiated. > The sequence goes this way : > > 1 Client -> Server : SYN > 2 Server -> Client : SYN/ACK > 3 Client -> Server : ACK > 4 Client -> Server : PSH/ACK (upper protocol data sending starts here) > 5 Server -> Client : RST > > - The problem happens sporadically, same client and same server can communicate smoothely on the same service port. But from time to time (hours, sometime days) the previous sequence happens. > - The service running on server is not responsible for the RST sent. The service was deeply profiled and nothing happens to justify the RST. > - tcpdump on the server side assures that packet arrives timely ordered. > - the traffic is very light. Some TCP sessions per day. > - the server is connected using a lagg enslaving two cxgb interfaces. > > In my effort to diagnose the problem (try to have a reproductible test case) I noticed that the issue is triggered most likely when those two conditions are met : > - the ACK (in step 3) and the PSH/ACK (in step 4) arrive on different lagg NICs. > - the timing between those two packets is sub 10 microseconds. > > When searching the interwebs I came across a strangely similar issue reported here 7 years ago : > https://lists.freebsd.org/pipermail/freebsd-net/2010-August/026029.html > > (The OP seemed to have resolved his issue changing the netisr policy from direct to hybrid. but no reference of laggs being used) > > I'm pretty sure that I'm hitting some race condition, a scenario where due to multithreading the PSH/ACK is somehow handled before the ACK making the kernel rising TCP/RST since the initial TCP handshake did'nt finish yet. > > I've read about netisr work and I was under the impression that even if it's SMP enabled it was made to keep prorocol ordering. > > What's the expected behaviour in this scenario on the netisr side ? > How can I push the investigation further ? I think you've already figured out the situation here -- the PSH/ACK is likely being handled before the ACK for the SYN because they arrived on different interfaces. There is nothing in netisr dispatch that will maintain protocol ordering in this case. Regards, Navdeep From owner-freebsd-net@freebsd.org Mon Jun 26 22:13:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9684BD92D4F for ; Mon, 26 Jun 2017 22:13:46 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BCA0683B5 for ; Mon, 26 Jun 2017 22:13:46 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id 62so9278170wmw.1 for ; Mon, 26 Jun 2017 15:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=27Btf0Fv07zuYCxQ9Uzw7YwBgpdaZeWPtbdR9DAsb9E=; b=lTkw8igVPe3cQUItDqlUYrVZjQrcrjJcwwMHH5JaEO0V/iwl+aflsAlYoVrv/1htC5 mmqpxtbc3S2WIQTk6dRf3tbp9j8B0K5CLxExs7lr5io01WnWkLOglls6+l7mZeeXT8o/ N+W096y4mdCsi5UxL9LpxmlaYgg5EmWpPrw8Jxl40iDxGSFJwxoYCyLcfNr+AxZbCvP1 GOrA3GOTtxqn1mOXAiQlB7mcEoM7KjWt1oP0Hjj0f3kmoZ1uRM+2+nFZ8Bel2pD9DBqo 8QZRY8CasYYCVaipFC2UJ6OdB93HnPjeCDEYepppqEMZFr43E0Y4wRITnOBMQw7WsRsc O3mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=27Btf0Fv07zuYCxQ9Uzw7YwBgpdaZeWPtbdR9DAsb9E=; b=YO9FvGL8ie1UNWVNYHYUExNTGI0ygol2IYMaoSs8v8REGh46Xu9dlp4WZJ2dE0Cqsp f0PruACnSQf3AZZCdeXmE0GscAxNCEpaKEeOrM4+9N9nICo4Y653A3DvZCBy3FBEDpl9 ZBZyvEZRnEs8A/RF2HUfj6j+upEKgDSopgQ5ybsSa6JM/wX7fnpvfY4+anA+TKxzKkgK 2mMwdJwNUZ/VrzdM43SNfLvX74RChvggRKw8+LhV92UL1QL2hpq7ZWVdVddnpYPD47ej IWGGOk9C2E1Na9VcxAnHC8acQ/WFREhFZC5PbgN9gCuag3OtoRRkzrxG6VQur90D01sp 8ePA== X-Gm-Message-State: AKS2vOxKqOW6G7JlYRMkS/TV6WfwuYOHGf0O+IjiXxLx76OTTmQMQwsj 9GIgB41qeXVGFH/qCjQzTXkaudW7kQ== X-Received: by 10.28.113.21 with SMTP id m21mr1154663wmc.80.1498515224120; Mon, 26 Jun 2017 15:13:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.160.42 with HTTP; Mon, 26 Jun 2017 15:13:43 -0700 (PDT) In-Reply-To: References: <84CB0795-B28E-46DF-9593-4C1BAAB7DDF5@pasteur.fr> From: Matt Joras Date: Mon, 26 Jun 2017 15:13:43 -0700 Message-ID: Subject: Re: Sporadic TCP/RST sent to client To: Navdeep Parhar Cc: Youssef GHORBAL , "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2017 22:13:46 -0000 Out of curiosity, what sort of lagg setup are you using that's causing the TCP packets to be split across the two lagg interfaces? Matt On Mon, Jun 26, 2017 at 1:35 PM, Navdeep Parhar wrote: > On Thu, Jun 22, 2017 at 3:57 PM, Youssef GHORBAL > wrote: >> Hello, >> >> I'm having an issue with a FreeBSD 11 based system, sending sporadically TCP/RST to clients after initial TCP session correctly initiated. >> The sequence goes this way : >> >> 1 Client -> Server : SYN >> 2 Server -> Client : SYN/ACK >> 3 Client -> Server : ACK >> 4 Client -> Server : PSH/ACK (upper protocol data sending starts here) >> 5 Server -> Client : RST >> >> - The problem happens sporadically, same client and same server can communicate smoothely on the same service port. But from time to time (hours, sometime days) the previous sequence happens. >> - The service running on server is not responsible for the RST sent. The service was deeply profiled and nothing happens to justify the RST. >> - tcpdump on the server side assures that packet arrives timely ordered. >> - the traffic is very light. Some TCP sessions per day. >> - the server is connected using a lagg enslaving two cxgb interfaces. >> >> In my effort to diagnose the problem (try to have a reproductible test case) I noticed that the issue is triggered most likely when those two conditions are met : >> - the ACK (in step 3) and the PSH/ACK (in step 4) arrive on different lagg NICs. >> - the timing between those two packets is sub 10 microseconds. >> >> When searching the interwebs I came across a strangely similar issue reported here 7 years ago : >> https://lists.freebsd.org/pipermail/freebsd-net/2010-August/026029.html >> >> (The OP seemed to have resolved his issue changing the netisr policy from direct to hybrid. but no reference of laggs being used) >> >> I'm pretty sure that I'm hitting some race condition, a scenario where due to multithreading the PSH/ACK is somehow handled before the ACK making the kernel rising TCP/RST since the initial TCP handshake did'nt finish yet. >> >> I've read about netisr work and I was under the impression that even if it's SMP enabled it was made to keep prorocol ordering. >> >> What's the expected behaviour in this scenario on the netisr side ? >> How can I push the investigation further ? > > I think you've already figured out the situation here -- the PSH/ACK is likely > being handled before the ACK for the SYN because they arrived on different > interfaces. There is nothing in netisr dispatch that will maintain protocol > ordering in this case. > > Regards, > Navdeep > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Tue Jun 27 05:13:22 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FA43DA0F91; Tue, 27 Jun 2017 05:13:22 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 911E87BDAB; Tue, 27 Jun 2017 05:13:21 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id b184so15089553wme.1; Mon, 26 Jun 2017 22:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mXHIc7XZRwI1tVFy4U5Y05uxFRGexqrwlKrlqodjHzQ=; b=gt06FYWtXnbbSbtYM0mL22+dmjpcOIyjXhwYTMigK7VZyiZw95g0tsF6c3jj2E4otS eBXiXymwMSSF1NfP7zNoGNmCUUKwqBFXbf05MBlVK278wQfa2qpoA1Y95HBTNeqXs27b yGMnoi7c3jtirbCQ7Yqv14OHMPwcaZSuamfLnlX7qwXqIMdNYhwTAjqY209Ud40T7atr oGEIJEeKzy/HzYOyJuE0qhqeBN/gxxffUAInsPq73cdSkjuY9ztTPm29d2FK+VSlkCHf 3/ASf7yuP6sygy5pqVr6S4Vjds/LQi6kSXYmmBTJOKaVywwmFWK8lNkXpz8NX/Er5ruv HYpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mXHIc7XZRwI1tVFy4U5Y05uxFRGexqrwlKrlqodjHzQ=; b=grHq/+mrV2SgKsu7J9xQy9WLaWGyTONi+ZMRSJOe8cCGzH9FFuIDHrxH4W2IksaHK6 hTi3XUFv4m4ts0BZUlKSshK2UaEjijvbxr8F+hGaifC9k10h6qyomFaMyzJ02S1zVDqs op9VK6ZMSjsQP4Iehg5LaHsVCBhRUJCrDEV0pEgORbG70R4GZTuEgp+pSSnFuf+wB9bW P2IzsbZcD+9QDxWlzyKOvxvG+mQWBe2dBrlFS8cOkrDDyCaPwCPBHIXrsQlpgnwZIPVC PBmF30gCT3nUMAoXNUHX19a1H9lLAFSreDMgpV/y5U7D1Fsh0dPiiMphjZxi5mgkiM3R JLjw== X-Gm-Message-State: AKS2vOxIFQ1anzdM7Bxn/6U7/R4PZc+VQLe5qnDNFHHDpRGvt/YgCnS7 rR1M2Q/t+1PWplq/l7ADtwbJiVK42A== X-Received: by 10.80.161.69 with SMTP id 63mr2469430edj.142.1498540399677; Mon, 26 Jun 2017 22:13:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.183.176 with HTTP; Mon, 26 Jun 2017 22:13:18 -0700 (PDT) In-Reply-To: <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> From: Zaphod Beeblebrox Date: Tue, 27 Jun 2017 01:13:18 -0400 Message-ID: Subject: Re: mbuf_jumbo_9k & iSCSI failing To: Ben RUBSON Cc: Ryan Stone , FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 05:13:22 -0000 Don't forget that, generally, as I understand it, the network stack suffers from the same problem for 9k buffers. On Sun, Jun 25, 2017 at 12:56 PM, Ben RUBSON wrote: > > On 25 Jun 2017, at 17:32, Ryan Stone wrote: > > > > Having looking at the original email more closely, I see that you showed > an mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > clusters increase while you are far below the zone's limit means that > you're definitely running into the bug I'm describing, and this bug could > plausibly cause the iSCSI errors that you describe. > > > > The issue is that the newer version of the driver tries to allocate a > single buffer to accommodate an MTU-sized packet. Over time, however, > memory will become fragmented and eventually it can become impossible to > allocate a 9k physically contiguous buffer. When this happens the driver > is unable to allocate buffers to receive packets and is forced to drop > them. Presumably, if iSCSI suffers too many packet drops it will terminate > the connection. The older version of the driver limited itself to > page-sized buffers, so it was immune to issues with memory fragmentation. > > Thank you for your explanation Ryan. > You say "over time", and you're right, I have to wait several days (here > 88) before the problem occurs. > Strange however that in 2500MB free memory system is unable to find 9k > physically contiguous. But we never know :) > > Let's then wait for your patch ! > (and reboot for now) > > Many thx ! > > Ben > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Tue Jun 27 09:04:36 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D534BDA55A1 for ; Tue, 27 Jun 2017 09:04:36 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Received: from mx0.pasteur.fr (mx0.pasteur.fr [157.99.45.50]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "Cisco Appliance Demo Certificate", Issuer "Cisco Appliance Demo Certificate" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 28B79829D2 for ; Tue, 27 Jun 2017 09:04:35 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Authentication-Results: mx0.pasteur.fr; spf=None smtp.pra=youssef.ghorbal@pasteur.fr; spf=None smtp.mailfrom=youssef.ghorbal@pasteur.fr; spf=None smtp.helo=postmaster@EXCHANGE02.corp.pasteur.fr Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=pra; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=mailfrom; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of postmaster@EXCHANGE02.corp.pasteur.fr) identity=helo; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="postmaster@EXCHANGE02.corp.pasteur.fr"; x-conformance=sidf_compatible X-IronPort-AV: E=Sophos;i="5.39,399,1493676000"; d="scan'208";a="1442675" Received: from exchange02.corp.pasteur.fr ([157.99.211.32]) by mx0.pasteur.fr with ESMTP/TLS/AES256-GCM-SHA384; 27 Jun 2017 11:04:27 +0200 Received: from EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) by EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.845.34; Tue, 27 Jun 2017 11:04:26 +0200 Received: from EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20]) by EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20%18]) with mapi id 15.01.0845.034; Tue, 27 Jun 2017 11:04:26 +0200 From: "Youssef GHORBAL" To: Navdeep Parhar CC: "freebsd-net@freebsd.org" Subject: Re: Sporadic TCP/RST sent to client Thread-Topic: Sporadic TCP/RST sent to client Thread-Index: AQHS66rb2Qoho5P9iUa0cYMOeRPraqI3fpaAgADRTIA= Date: Tue, 27 Jun 2017 09:04:26 +0000 Message-ID: References: <84CB0795-B28E-46DF-9593-4C1BAAB7DDF5@pasteur.fr> In-Reply-To: Accept-Language: en-US, fr-FR Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [157.99.101.113] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 09:04:36 -0000 [...] >> I've read about netisr work and I was under the impression that e= ven if it's SMP enabled it was made to keep prorocol ordering. >>=20 >> What's the expected behaviour in this scenario on the netisr side= ? >> How can I push the investigation further ? >=20 > I think you've already figured out the situation here -- the PSH/ACK is l= ikely > being handled before the ACK for the SYN because they arrived on differen= t > interfaces. There is nothing in netisr dispatch that will maintain proto= col > ordering in this case. Navdeep, thank you for you feedback. I don't get the fact that netisr is no= t lagg "aware" (if I may say) I understand that netisr dispatch can't maintain ordering if NICs are treat= ed separetly. But when they are enslaved in a "lagg", from the point of the= view of the lagg interface itself, packets arrive ordred and I expect the = system to handle it correctly. The fact is that it is actually handled corr= ectly in most cases but not when packets are really "close" (under 10 micro= seconds) Maybe the default sysctl settings are not meant to handle this corner case,= but I did'nt find anything regarding this in documentation. Youssef Ghorbal= From owner-freebsd-net@freebsd.org Tue Jun 27 09:15:40 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7601BDA5838 for ; Tue, 27 Jun 2017 09:15:40 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Received: from mx0.pasteur.fr (mx0.pasteur.fr [157.99.45.50]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "Cisco Appliance Demo Certificate", Issuer "Cisco Appliance Demo Certificate" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D45282E17 for ; Tue, 27 Jun 2017 09:15:39 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Authentication-Results: mx0.pasteur.fr; spf=None smtp.pra=youssef.ghorbal@pasteur.fr; spf=None smtp.mailfrom=youssef.ghorbal@pasteur.fr; spf=None smtp.helo=postmaster@EXCHANGE02.corp.pasteur.fr Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=pra; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=mailfrom; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of postmaster@EXCHANGE02.corp.pasteur.fr) identity=helo; client-ip=157.99.211.32; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="postmaster@EXCHANGE02.corp.pasteur.fr"; x-conformance=sidf_compatible X-IronPort-AV: E=Sophos;i="5.39,399,1493676000"; d="scan'208";a="1443271" Received: from exchange02.corp.pasteur.fr ([157.99.211.32]) by mx0.pasteur.fr with ESMTP/TLS/AES256-GCM-SHA384; 27 Jun 2017 11:15:37 +0200 Received: from EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) by EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.845.34; Tue, 27 Jun 2017 11:15:36 +0200 Received: from EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20]) by EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20%18]) with mapi id 15.01.0845.034; Tue, 27 Jun 2017 11:15:36 +0200 From: "Youssef GHORBAL" To: Matt Joras CC: Navdeep Parhar , "freebsd-net@freebsd.org" Subject: Re: Sporadic TCP/RST sent to client Thread-Topic: Sporadic TCP/RST sent to client Thread-Index: AQHS66rb2Qoho5P9iUa0cYMOeRPraqI3fpaAgAAbfoCAALjuAA== Date: Tue, 27 Jun 2017 09:15:36 +0000 Message-ID: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> References: <84CB0795-B28E-46DF-9593-4C1BAAB7DDF5@pasteur.fr> In-Reply-To: Accept-Language: en-US, fr-FR Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [157.99.101.113] Content-Type: text/plain; charset="us-ascii" Content-ID: <4270729117D59544BAA4C524A37127EE@corp.pasteur.fr> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 09:15:40 -0000 Imagine this set up : freebsd host port0 <-> switch 1 <-> linux host port0 freebsd host port1 <-> switch 2 <-> linux host port1 On the linux box, port 0&1 are enslaved in a bond with a RR algorithm (Roun= d Robin) On the freebsd box, port 0&1 are enslaved in a lagg. switchs 1&2 are configured for doing MLAG. The Linux box disapatchs packets on both NICs (since the RR algo dictates t= hat) packets are dispatched in order. Packets outgoing on port0 gets handled by switch1 and hits the freebsd box = on port 0 Packets outgoing on port1 gets handled by switch2 and hits the freebsd box = on port 1 As I stated earlier, from the tcpdump traces I've done on the freebsd box (= both on the lagg interface and the actual ports) packets do arrive ordered = but on different NICs and it works great until the elapes times start to be= around microsecond. I don't really have control over the Linux box to make them use other hash = algo (but I'm stil trying) Youssef ------------------------ > On 27 Jun 2017, at 00:13, Matt Joras wrote: >=20 > Out of curiosity, what sort of lagg setup are you using that's causing > the TCP packets to be split across the two lagg interfaces? >=20 > Matt >=20 > On Mon, Jun 26, 2017 at 1:35 PM, Navdeep Parhar wrote= : >> On Thu, Jun 22, 2017 at 3:57 PM, Youssef GHORBAL >> wrote: >>> Hello, >>>=20 >>> I'm having an issue with a FreeBSD 11 based system, sending spor= adically TCP/RST to clients after initial TCP session correctly initiated. >>> The sequence goes this way : >>>=20 >>> 1 Client -> Server : SYN >>> 2 Server -> Client : SYN/ACK >>> 3 Client -> Server : ACK >>> 4 Client -> Server : PSH/ACK (upper protocol data sending starts= here) >>> 5 Server -> Client : RST >>>=20 >>> - The problem happens sporadically, same client and same server = can communicate smoothely on the same service port. But from time to time (= hours, sometime days) the previous sequence happens. >>> - The service running on server is not responsible for the RST s= ent. The service was deeply profiled and nothing happens to justify the RST= . >>> - tcpdump on the server side assures that packet arrives timely = ordered. >>> - the traffic is very light. Some TCP sessions per day. >>> - the server is connected using a lagg enslaving two cxgb interf= aces. >>>=20 >>> In my effort to diagnose the problem (try to have a reproductibl= e test case) I noticed that the issue is triggered most likely when those t= wo conditions are met : >>> - the ACK (in step 3) and the PSH/ACK (in step 4) arrive on diff= erent lagg NICs. >>> - the timing between those two packets is sub 10 microseconds. >>>=20 >>> When searching the interwebs I came across a strangely similar i= ssue reported here 7 years ago : >>> https://lists.freebsd.org/pipermail/freebsd-net/2010-August/0260= 29.html >>>=20 >>> (The OP seemed to have resolved his issue changing the netisr po= licy from direct to hybrid. but no reference of laggs being used) >>>=20 >>> I'm pretty sure that I'm hitting some race condition, a scenario= where due to multithreading the PSH/ACK is somehow handled before the ACK = making the kernel rising TCP/RST since the initial TCP handshake did'nt fin= ish yet. >>>=20 >>> I've read about netisr work and I was under the impression that = even if it's SMP enabled it was made to keep prorocol ordering. >>>=20 >>> What's the expected behaviour in this scenario on the netisr sid= e ? >>> How can I push the investigation further ? >>=20 >> I think you've already figured out the situation here -- the PSH/ACK is = likely >> being handled before the ACK for the SYN because they arrived on differe= nt >> interfaces. There is nothing in netisr dispatch that will maintain prot= ocol >> ordering in this case. >>=20 >> Regards, >> Navdeep >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Tue Jun 27 10:54:29 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F424DA70A0 for ; Tue, 27 Jun 2017 10:54:29 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from bizet.nethelp.no (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by mx1.freebsd.org (Postfix) with ESMTP id 06B55FC6 for ; Tue, 27 Jun 2017 10:54:28 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from localhost (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by bizet.nethelp.no (Postfix) with ESMTP id 4003BE6065; Tue, 27 Jun 2017 12:54:26 +0200 (CEST) Date: Tue, 27 Jun 2017 12:54:26 +0200 (CEST) Message-Id: <20170627.125426.74697078.sthaug@nethelp.no> To: youssef.ghorbal@pasteur.fr Cc: matt.joras@gmail.com, freebsd-net@freebsd.org, nparhar@gmail.com Subject: Re: Sporadic TCP/RST sent to client From: sthaug@nethelp.no In-Reply-To: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> References: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 10:54:29 -0000 > Imagine this set up : > > freebsd host port0 <-> switch 1 <-> linux host port0 > freebsd host port1 <-> switch 2 <-> linux host port1 > > On the linux box, port 0&1 are enslaved in a bond with a RR algorithm (Round Robin) > On the freebsd box, port 0&1 are enslaved in a lagg. > > switchs 1&2 are configured for doing MLAG. > > The Linux box disapatchs packets on both NICs (since the RR algo dictates that) packets are dispatched in order. > Packets outgoing on port0 gets handled by switch1 and hits the freebsd box on port 0 > Packets outgoing on port1 gets handled by switch2 and hits the freebsd box on port 1 > > As I stated earlier, from the tcpdump traces I've done on the freebsd box (both on the lagg interface and the actual ports) packets do arrive ordered but on different NICs and it works great until the elapes times start to be around microsecond. > > I don't really have control over the Linux box to make them use other hash algo (but I'm stil trying) If the Linux box is using round robin you shouldn't expect to be able to "fix" the problem at the FreeBSD end. On routers and switches (which is what I normally work with) the hash algorithm used for LAG connections ensures that one "flow" always uses the same path, thus no reordering. A typical hash algorithm uses a 5-tuple with (src ip, src port, dst ip, dst port, protocol) as input. So the advice in this case is simple - don't use round robin! Yes, I understand you don't control the Linux box. Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@freebsd.org Tue Jun 27 12:05:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CF33DA8E8B for ; Tue, 27 Jun 2017 12:05:10 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Received: from mx0.pasteur.fr (mx0.pasteur.fr [157.99.45.50]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "Cisco Appliance Demo Certificate", Issuer "Cisco Appliance Demo Certificate" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0677638F8 for ; Tue, 27 Jun 2017 12:05:09 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Authentication-Results: mx0.pasteur.fr; spf=None smtp.pra=youssef.ghorbal@pasteur.fr; spf=None smtp.mailfrom=youssef.ghorbal@pasteur.fr; spf=None smtp.helo=postmaster@EXCHANGE04.corp.pasteur.fr Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=pra; client-ip=157.99.211.34; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=mailfrom; client-ip=157.99.211.34; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of postmaster@EXCHANGE04.corp.pasteur.fr) identity=helo; client-ip=157.99.211.34; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="postmaster@EXCHANGE04.corp.pasteur.fr"; x-conformance=sidf_compatible X-IronPort-AV: E=Sophos;i="5.39,399,1493676000"; d="scan'208";a="1450646" Received: from exchange04.corp.pasteur.fr ([157.99.211.34]) by mx0.pasteur.fr with ESMTP/TLS/AES256-GCM-SHA384; 27 Jun 2017 14:05:07 +0200 Received: from EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) by EXCHANGE04.corp.pasteur.fr (2002:9d63:d322::9d63:d322) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.845.34; Tue, 27 Jun 2017 14:05:06 +0200 Received: from EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20]) by EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20%18]) with mapi id 15.01.0845.034; Tue, 27 Jun 2017 14:05:06 +0200 From: "Youssef GHORBAL" To: "sthaug@nethelp.no" CC: "matt.joras@gmail.com" , "freebsd-net@freebsd.org" , "nparhar@gmail.com" Subject: Re: Sporadic TCP/RST sent to client Thread-Topic: Sporadic TCP/RST sent to client Thread-Index: AQHS66rb2Qoho5P9iUa0cYMOeRPraqI3fpaAgAAbfoCAALjuAIAAG50AgAATvwA= Date: Tue, 27 Jun 2017 12:05:06 +0000 Message-ID: References: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> <20170627.125426.74697078.sthaug@nethelp.no> In-Reply-To: <20170627.125426.74697078.sthaug@nethelp.no> Accept-Language: en-US, fr-FR Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [157.99.101.110] Content-Type: text/plain; charset="us-ascii" Content-ID: <9A4CBCD005B2534E88EEE4CC29B5949A@corp.pasteur.fr> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 12:05:10 -0000 > On 27 Jun 2017, at 12:54, sthaug@nethelp.no wrote: >=20 >> Imagine this set up : >>=20 >> freebsd host port0 <-> switch 1 <-> linux host port0 >> freebsd host port1 <-> switch 2 <-> linux host port1 >>=20 >> On the linux box, port 0&1 are enslaved in a bond with a RR algorithm (R= ound Robin) >> On the freebsd box, port 0&1 are enslaved in a lagg. >>=20 >> switchs 1&2 are configured for doing MLAG. >>=20 >> The Linux box disapatchs packets on both NICs (since the RR algo dictate= s that) packets are dispatched in order. >> Packets outgoing on port0 gets handled by switch1 and hits the freebsd b= ox on port 0 >> Packets outgoing on port1 gets handled by switch2 and hits the freebsd b= ox on port 1 >>=20 >> As I stated earlier, from the tcpdump traces I've done on the freebsd bo= x (both on the lagg interface and the actual ports) packets do arrive order= ed but on different NICs and it works great until the elapes times start to= be around microsecond. >>=20 >> I don't really have control over the Linux box to make them use other ha= sh algo (but I'm stil trying) >=20 > If the Linux box is using round robin you shouldn't expect to be able > to "fix" the problem at the FreeBSD end. There is nothing in the 802.3ad that mandates stickiness of flows per NIC, = the only thing explicit is that hash algorithm needs to maintain packet ord= er. In this case, strictly speaking, it's : Packets do leave in "order" and= do arrive in "order". > On routers and switches (which is what I normally work with) the hash > algorithm used for LAG connections ensures that one "flow" always uses > the same path, thus no reordering. A typical hash algorithm uses a > 5-tuple with (src ip, src port, dst ip, dst port, protocol) as input. >=20 > So the advice in this case is simple - don't use round robin! Yes, I > understand you don't control the Linux box. Sure, I was just wondering if the FreeBSD network stack was built with the = fact that each flow needs to arrive on the same NIC and the system was desi= gned with this assumption in mind or not. I reported it here, thinking that maybe it's a subtle buggy corner case and= maybe the community was interesting to know about and maybe fix : - If the stack is working as expected and was built with the assumption tha= t each incoming flow needs to stick to a NIC during it's lifetime, maybe do= cumentation needs to be more explicit regarding this situation. In that cas= e I'll file documentation enhancement bug report. - If the stack is misbehaving, maybe help the community identify the root c= ause and help fixing it Youssef From owner-freebsd-net@freebsd.org Tue Jun 27 12:46:24 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F333EDA9C4E for ; Tue, 27 Jun 2017 12:46:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E0AD564B6D for ; Tue, 27 Jun 2017 12:46:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RCkOfN090524 for ; Tue, 27 Jun 2017 12:46:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 12:46:25 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 12:46:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #78 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #77) Hi guys, After 8 days working with no issues i think at last it has been fixed. Well done Eugene :) I could see libc patch has been committed to 11-STABLE. Is there a plan to commit libthr patch and all others PRs related before 11.1-RELEASE? Thank you guys. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:06:13 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65BF4D8B0D0 for ; Tue, 27 Jun 2017 17:06:13 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-wr0-x22d.google.com (mail-wr0-x22d.google.com [IPv6:2a00:1450:400c:c0c::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC3C7745E6 for ; Tue, 27 Jun 2017 17:06:12 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-wr0-x22d.google.com with SMTP id c11so162505359wrc.3 for ; Tue, 27 Jun 2017 10:06:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=VS3RBIjlRRAQdaYcuaJOcQ7doSjInLpuu2d+3569afo=; b=qZ6r7GW+gWhXMorw3YKY0mcYhdaVeCg5MGghTq8kQULjJUdIFgtW5vr/miOwU9hqBX oqoyJWoLu6QOJtuM6gFpX4pZkg74ouk3oCTaphtvxD6v/5Twmx5ZdCFFKkpkBCHRWzSK KaBxAvsaMViLN0a0MO/iKeJ2Woetmq4a2qx0ramF9xMWBfFMAFwaFDRCNiLR7z/vJchU RSWFwg06KFfhjD4xrWNZ3MDQFZNrerFjsuhCiSLfLWxhH9bXwzihHrxllegfXhwxY0dp z9jno7G7ok7eQhtjP6p2uZjrRVCoeiv3fLO3t48NJPTBYF7lmklMaTc2s07BXJpThhnk xKXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=VS3RBIjlRRAQdaYcuaJOcQ7doSjInLpuu2d+3569afo=; b=Co2X4wfGXtpgpCR6xrArXZnO81iboHzv3lC641QCpztwb3oneA9dDxY2f1MOHTo2rW R7Hh9FkgbwN+wviUAkipn21+6h0nZUtlH+3X7Gvw1B0mU4jVtpzvWDGO/kVnvibg0I/2 MmCEIVx0tbnTGZN8jjtTSc8V95s/Es/a4nZ2TVDYHovVsitTBlqSSREx558kf7ibOPxV elyM4/rG6LKUZA0O4mEwvrm0HuE4IhnwcbD7ib8zq1wIGEWo62goB1oROhTMqRNkrojj Za8y3f0OBGCQU6RH5/oDrS/QHQGn9e2/8EPWkNPgbGFW6alemH0JZZrkPbqZa1p3hVCE JMpA== X-Gm-Message-State: AKS2vOzpm1yEtRN4lPkcgl3zgOHIE2Y7G8aKI5sLiER1bOsaLmvaXTgD 9WdNQt51BQpoew3ndx5/W3BOFCx5iA== X-Received: by 10.223.143.77 with SMTP id p71mr17134911wrb.3.1498583171183; Tue, 27 Jun 2017 10:06:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.160.42 with HTTP; Tue, 27 Jun 2017 10:06:10 -0700 (PDT) In-Reply-To: References: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> <20170627.125426.74697078.sthaug@nethelp.no> From: Matt Joras Date: Tue, 27 Jun 2017 10:06:10 -0700 Message-ID: Subject: Re: Sporadic TCP/RST sent to client To: Youssef GHORBAL Cc: "sthaug@nethelp.no" , "freebsd-net@freebsd.org" , "nparhar@gmail.com" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:06:13 -0000 On Tue, Jun 27, 2017 at 5:05 AM, Youssef GHORBAL wrote: > > There is nothing in the 802.3ad that mandates stickiness of flows per NIC= , the only thing explicit is that hash algorithm needs to maintain packet o= rder. In this case, strictly speaking, it's : Packets do leave in "order" a= nd do arrive in "order". I think the important point is that the ordering is not guaranteed in this case, despite whether it's happening or not. As soon as you are using a round-robin lagg on one end you've pretty much lost all guarantees of ordering at the remote end. Unless the switch has some way to know, which as Steinar noted is usually done through a negotiated or statically-configured hash-based lagg, there's no way for it to enforce the ordering you're expecting for proper behaviour. So even if there was some notion of protocol ordering in netisr, the fact that you're using round-robin on one endpoint opens up the possibility for this kind of situation anyway. Further, I would argue that round robin is not a valid 802.3ad/802.1AX algorithm, per how it defines a frame distributor: "This standard does not mandate any particular distribution algorithm(s); however, any distribution algorithm shall ensure that, when frames are received by a Frame Collector as specified in 5.2.3, the algorithm shall not cause: a) Misordering of frames that are part of any given conversation, or b) Duplication of frames. The above requirement to maintain frame ordering is met by ensuring that all frames that compose a given conversation are transmitted on a single link in the order that they are generated by the MAC Client; hence, this requirement does not involve the addition (or modification) of any information to the MAC frame, nor any buffering or processing on the part of the corresponding Frame Collector in order to reorder frames." > Sure, I was just wondering if the FreeBSD network stack was built with th= e fact that each flow needs to arrive on the same NIC and the system was de= signed with this assumption in mind or not. > > I reported it here, thinking that maybe it's a subtle buggy corner case a= nd maybe the community was interesting to know about and maybe fix : > > - If the stack is working as expected and was built with the assumption t= hat each incoming flow needs to stick to a NIC during it's lifetime, maybe = documentation needs to be more explicit regarding this situation. In that c= ase I'll file documentation enhancement bug report. > - If the stack is misbehaving, maybe help the community identify the root= cause and help fixing it > As far as I can tell, as Navdeep noted, there's no unexpected behaviour in your case. "Flows" are a concept that the protocols, in this case TCP, knows about. The devices themselves (Ethernet cards) usually have mechanics to make packet delivery decisions based on flow information (e.g. RSS hashing), but as far as I know that is generally limited within a single port, so it doesn't really help in the general case of a lagg. Matt From owner-freebsd-net@freebsd.org Tue Jun 27 17:23:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3746BD8B75A for ; Tue, 27 Jun 2017 17:23:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 255A67537F for ; Tue, 27 Jun 2017 17:23:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHMwmX087635 for ; Tue, 27 Jun 2017 17:23:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 17:22:59 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:23:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #79 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #78) I believe, the libthr patch was for debugging purposes only and is not need= ed to fix the problem itself. Did you test it with web server disabled? If so, please update your ports t= ree and rebuild mpd5 port that now has "console" patch integrated, so remove yo= ur local version of the patch before updating ports tree - my tests show that = it should be enough to fix mpd's web server too. And run mpd-5.8_1 with web se= rver enabled as you did earlier to verify that. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:28:16 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF9A7D8B9AC for ; Tue, 27 Jun 2017 17:28:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE1A5757CD for ; Tue, 27 Jun 2017 17:28:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHSGVL095314 for ; Tue, 27 Jun 2017 17:28:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 146037] [panic] mpd + CoA = kernel panic Date: Tue, 27 Jun 2017 17:28:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Feedback Timeout X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:28:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D146037 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Feedback Timeout Status|Open |Closed --- Comment #3 from Eugene Grosbein --- Feedback timeout. Please try FreeBSD 11-STABLE if you still have this probl= em. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:38:32 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A95AD8BF8F for ; Tue, 27 Jun 2017 17:38:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2825675FFE for ; Tue, 27 Jun 2017 17:38:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHcVX6019070 for ; Tue, 27 Jun 2017 17:38:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 199096] Kernel panic after some time using mpd (netgraph) and ipfw Date: Tue, 27 Jun 2017 17:38:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Feedback Timeout X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:38:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D199096 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Feedback Timeout Status|Open |Closed --- Comment #4 from Eugene Grosbein --- Feedback timeout. Please try any supported FreeBSD version if you still have this problem. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:46:30 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32FF4D8C546 for ; Tue, 27 Jun 2017 17:46:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 209B576AD5 for ; Tue, 27 Jun 2017 17:46:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHkSf6039279 for ; Tue, 27 Jun 2017 17:46:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 17:46:28 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:46:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #80 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #79) Yes, i did. All my tests were with web server disabled as you requested. On= ly with console enabled. I'll run with web server enabled to try the patch.=20 How about other related patchs (like netgraph, ipfw, etc) ? Are you going to commit? Thanks. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:53:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD4E4D8C894 for ; Tue, 27 Jun 2017 17:53:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAFDC770A7 for ; Tue, 27 Jun 2017 17:53:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHrTXS058140 for ; Tue, 27 Jun 2017 17:53:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 17:53:29 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:53:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #81 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #80) I cannot commit kernel patches myself as I have no src commit bit. Any src committed is needed to take a look at least, so I've filled my PRs. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 17:59:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8904DD8CCD3 for ; Tue, 27 Jun 2017 17:59:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7702F774DD for ; Tue, 27 Jun 2017 17:59:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RHxYIp066506 for ; Tue, 27 Jun 2017 17:59:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 17:59:35 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 17:59:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #82 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #81) Humm i see. Maybe Konstantin Belousov could help us :) --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 18:31:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75852D8D796 for ; Tue, 27 Jun 2017 18:31:10 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Received: from mx0.pasteur.fr (mx0.pasteur.fr [157.99.45.50]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "Cisco Appliance Demo Certificate", Issuer "Cisco Appliance Demo Certificate" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CF84E78428 for ; Tue, 27 Jun 2017 18:31:09 +0000 (UTC) (envelope-from prvs=34432f271=youssef.ghorbal@pasteur.fr) Authentication-Results: mx0.pasteur.fr; spf=None smtp.pra=youssef.ghorbal@pasteur.fr; spf=None smtp.mailfrom=youssef.ghorbal@pasteur.fr; spf=None smtp.helo=postmaster@EXCHANGE01.corp.pasteur.fr Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=pra; client-ip=157.99.211.31; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of youssef.ghorbal@pasteur.fr) identity=mailfrom; client-ip=157.99.211.31; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="youssef.ghorbal@pasteur.fr"; x-conformance=sidf_compatible Received-SPF: None (mx0.pasteur.fr: no sender authenticity information available from domain of postmaster@EXCHANGE01.corp.pasteur.fr) identity=helo; client-ip=157.99.211.31; receiver=mx0.pasteur.fr; envelope-from="youssef.ghorbal@pasteur.fr"; x-sender="postmaster@EXCHANGE01.corp.pasteur.fr"; x-conformance=sidf_compatible X-IronPort-AV: E=Sophos;i="5.40,271,1496095200"; d="scan'208";a="1463193" Received: from exchange01.corp.pasteur.fr ([157.99.211.31]) by mx0.pasteur.fr with ESMTP/TLS/AES256-GCM-SHA384; 27 Jun 2017 20:31:06 +0200 Received: from EXCHANGE02.corp.pasteur.fr (2002:9d63:d320::9d63:d320) by EXCHANGE01.corp.pasteur.fr (2002:9d63:d31f::9d63:d31f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.845.34; Tue, 27 Jun 2017 20:31:06 +0200 Received: from EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20]) by EXCHANGE02.corp.pasteur.fr ([fe80::a819:199f:2049:3d20%18]) with mapi id 15.01.0845.034; Tue, 27 Jun 2017 20:31:05 +0200 From: "Youssef GHORBAL" To: Matt Joras CC: "sthaug@nethelp.no" , "freebsd-net@freebsd.org" , "nparhar@gmail.com" Subject: Re: Sporadic TCP/RST sent to client Thread-Topic: Sporadic TCP/RST sent to client Thread-Index: AQHS66rb2Qoho5P9iUa0cYMOeRPraqI3fpaAgAAbfoCAALjuAIAAG50AgAATvwCAAFQdAIAAF7qA Date: Tue, 27 Jun 2017 18:31:04 +0000 Message-ID: References: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> <20170627.125426.74697078.sthaug@nethelp.no> In-Reply-To: Accept-Language: en-US, fr-FR Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [157.99.101.117] Content-Type: text/plain; charset="us-ascii" Content-ID: <58054922E07A124C9EE59E49BCCA74CF@corp.pasteur.fr> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 18:31:10 -0000 [...] > Further, I would argue that round robin is not a valid 802.3ad/802.1AX > algorithm, per how it defines a frame distributor: >=20 > "This standard does not mandate any particular distribution > algorithm(s); however, any distribution algorithm shall ensure that, > when frames are received by a Frame Collector as specified in 5.2.3, > the algorithm shall not cause: > a) Misordering of frames that are part of any given conversation, or > b) Duplication of frames. >=20 > The above requirement to maintain frame ordering is met by ensuring > that all frames that compose a given conversation are transmitted on a > single link in the order that they are generated by the MAC Client; > hence, this requirement does not involve the addition (or > modification) of any information to the MAC frame, nor any buffering > or processing on the part of the corresponding Frame Collector in > order to reorder frames." I tend to agree with you on this point. >> Sure, I was just wondering if the FreeBSD network stack was built with t= he fact that each flow needs to arrive on the same NIC and the system was d= esigned with this assumption in mind or not. >>=20 >> I reported it here, thinking that maybe it's a subtle buggy corner case = and maybe the community was interesting to know about and maybe fix : >>=20 >> - If the stack is working as expected and was built with the assumption = that each incoming flow needs to stick to a NIC during it's lifetime, maybe= documentation needs to be more explicit regarding this situation. In that = case I'll file documentation enhancement bug report. >> - If the stack is misbehaving, maybe help the community identify the roo= t cause and help fixing it >>=20 > As far as I can tell, as Navdeep noted, there's no unexpected > behaviour in your case. "Flows" are a concept that the protocols, in > this case TCP, knows about. The devices themselves (Ethernet cards) > usually have mechanics to make packet delivery decisions based on flow > information (e.g. RSS hashing), but as far as I know that is generally > limited within a single port, so it doesn't really help in the general > case of a lagg. So the fact that it works most of the time is just a "happy" coincidence. B= ut it's not a behaviour to relay on. Right ? Anyway, thank you very much for your help and the clarification on this iss= ue. Youssef= From owner-freebsd-net@freebsd.org Tue Jun 27 19:38:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A77A1D8EA65 for ; Tue, 27 Jun 2017 19:38:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 94FD97A5EC for ; Tue, 27 Jun 2017 19:38:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RJckes061433 for ; Tue, 27 Jun 2017 19:38:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220078] [patch] [panic] [ipfw] repeatable kernel panic due to unlocked INADDR_TO_IFP usage Date: Tue, 27 Jun 2017 19:38:46 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-ipfw@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 19:38:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220078 --- Comment #16 from Eugene Grosbein --- (In reply to Andrey V. Elsukov from comment #4) Andrey, there is no problems with your patch for ipfw. Please commit. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jun 27 21:14:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88E4AD907A7 for ; Tue, 27 Jun 2017 21:14:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 68FBB7D758 for ; Tue, 27 Jun 2017 21:14:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5RLEB1G048465 for ; Tue, 27 Jun 2017 21:14:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Tue, 27 Jun 2017 21:14:11 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2017 21:14:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #83 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #81) Eugene and Konstantin, Bad news, it just stopped working. Eugene i hadn't enabled web server yet. = So it stucked the mpd5 process with patch applied listed on comment #67. The top out i can see same message: 1295 root 9 52 0 314M 54820K uwrlck 7 625:46 0.74% mpd5 Here it is the bt: (gdb) bt #0 0x000000080228567c in _umtx_op_err () from /lib/libthr.so.3 #1 0x0000000802279c81 in __thr_rwlock_wrlock (rwlock=3D0x802494500, tsp=3D= ) at /usr/src/lib/libthr/thread/thr_umtx.c:325 #2 0x00000008022829f3 in _thr_rtld_wlock_acquire (lock=3D) at thr_umtx.h:239 #3 0x00000008006aec72 in wlock_acquire (lock=3D0x8008c3960, lockstate=3D0x7fffdfffda08) at /usr/src/libexec/rtld-elf/rtld_lock.c:217 #4 0x00000008006aaf4f in _rtld_allocate_tls (oldtls=3D0x0, tcbsize=3D32, tcbalign=3D16) at /usr/src/libexec/rtld-elf/rtld.c:4802 #5 0x0000000802281b69 in _tcb_ctor (thread=3D0x80c740f00, initial=3D) at /usr/src/lib/libthr/thread/thr_ctrdtr.c:45 #6 0x0000000802281110 in _thr_alloc (curthread=3D) at /usr/src/lib/libthr/thread/thr_list.c:170 #7 0x00000008022772a6 in _pthread_create (thread=3D0x80324d598, attr=3D0x0, start_routine=3D0x4653b0 , arg=3D) = at /usr/src/lib/libthr/thread/thr_create.c:78 #8 0x000000000046535b in ?? () #9 0x00000000004292a8 in ?? () #10 0x000000000042d219 in ?? () #11 0x0000000000448da3 in ?? () #12 0x00000000004493b1 in ?? () #13 0x000000000043ac94 in ?? () #14 0x000000000043d650 in ?? () #15 0x000000000043d366 in ?? () #16 0x000000000043b21d in ?? () #17 0x0000000000445012 in ?? () #18 0x000000000044b598 in ?? () #19 0x0000000000439c9f in ?? () #20 0x00000000004651d6 in ?? () #21 0x0000000000464908 in ?? () #22 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 Here it is all threads: (gdb) thread apply all bt Thread 9 (Thread 803016a00 (LWP 100704/)): #0 0x000000080228567c in _umtx_op_err () from /lib/libthr.so.3 #1 0x0000000802279c81 in __thr_rwlock_wrlock (rwlock=3D0x802494500, tsp=3D= ) at /usr/src/lib/libthr/thread/thr_umtx.c:325 #2 0x00000008022829f3 in _thr_rtld_wlock_acquire (lock=3D) at thr_umtx.h:239 #3 0x00000008006aec72 in wlock_acquire (lock=3D0x8008c3960, lockstate=3D0x7fffdfffda08) at /usr/src/libexec/rtld-elf/rtld_lock.c:217 #4 0x00000008006aaf4f in _rtld_allocate_tls (oldtls=3D0x0, tcbsize=3D32, tcbalign=3D16) at /usr/src/libexec/rtld-elf/rtld.c:4802 #5 0x0000000802281b69 in _tcb_ctor (thread=3D0x80c740f00, initial=3D) at /usr/src/lib/libthr/thread/thr_ctrdtr.c:45 #6 0x0000000802281110 in _thr_alloc (curthread=3D) at /usr/src/lib/libthr/thread/thr_list.c:170 #7 0x00000008022772a6 in _pthread_create (thread=3D0x80324d598, attr=3D0x0, start_routine=3D0x4653b0 , arg=3D) = at /usr/src/lib/libthr/thread/thr_create.c:78 #8 0x000000000046535b in ?? () #9 0x00000000004292a8 in ?? () #10 0x000000000042d219 in ?? () #11 0x0000000000448da3 in ?? () #12 0x00000000004493b1 in ?? () #13 0x000000000043ac94 in ?? () #14 0x000000000043d650 in ?? () #15 0x000000000043d366 in ?? () #16 0x000000000043b21d in ?? () #17 0x0000000000445012 in ?? () #18 0x000000000044b598 in ?? () #19 0x0000000000439c9f in ?? () #20 0x00000000004651d6 in ?? () #21 0x0000000000464908 in ?? () #22 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #23 0x0000000000000000 in ?? () Thread 8 (Thread 80326ca00 (LWP 101005/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fffc6731ed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fffc6532000 in ?? () Cannot access memory at address 0x7fffc6732000 Thread 7 (Thread 80326ed00 (LWP 101007/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fffb7cbced0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fffb7abd000 in ?? () Cannot access memory at address 0x7fffb7cbd000 ---Type to continue, or q to quit--- Thread 6 (Thread 803252e00 (LWP 100809/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fffb86c1ed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fffb84c2000 in ?? () Cannot access memory at address 0x7fffb86c2000 Thread 5 (Thread 807292a00 (LWP 100290/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fff88b44ed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fff88945000 in ?? () Cannot access memory at address 0x7fff88b45000 Thread 4 (Thread 803253d00 (LWP 101008/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fffbdaebed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fffbd8ec000 in ?? () Cannot access memory at address 0x7fffbdaec000 Thread 3 (Thread 808089000 (LWP 101010/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fff8bf5eed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fff8bd5f000 in ?? () Cannot access memory at address 0x7fff8bf5f000 Thread 2 (Thread 80808a400 (LWP 101011/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fff9178aed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () ---Type to continue, or q to quit--- #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fff9158b000 in ?? () Cannot access memory at address 0x7fff9178b000 Thread 1 (Thread 807258d00 (LWP 101012/)): #0 0x000000080257d43a in _nanosleep () from /lib/libc.so.7 #1 0x000000080227a4cc in __thr_nanosleep (time_to_sleep=3D0x7fff7c2e0ed0, time_remaining=3D0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:257 #2 0x00000008025d5076 in __usleep (useconds=3D) at /usr/src/lib/libc/gen/usleep.c:52 #3 0x0000000000465532 in ?? () #4 0x0000000802285550 in __pthread_cleanup_pop_imp (execute=3D) at /usr/src/lib/libthr/thread/thr_clean.c:73 #5 0x000000000046541e in ?? () #6 0x0000000802277b55 in thread_start (curthread=3D) = at /usr/src/lib/libthr/thread/thr_create.c:289 #7 0x00007fff7c0e1000 in ?? () Cannot access memory at address 0x7fff7c2e1000 #0 0x000000080228567c in _umtx_op_err () from /lib/libthr.so.3 Printof of *rwlock: (gdb) frame 1 #1 0x0000000802279c81 in __thr_rwlock_wrlock (rwlock=3D0x802494500, tsp=3D= ) at /usr/src/lib/libthr/thread/thr_umtx.c:325 325 return (_umtx_op_err(rwlock, UMTX_OP_RW_WRLOCK, 0, (void *)tm_size, Current language: auto; currently minimal (gdb) p *rwlock $1 =3D {rw_state =3D -1610612736, rw_flags =3D 2, rw_blocked_readers =3D 0, rw_blocked_writers =3D 0, rw_wowner =3D 100704, rw_spare =3D 0x802494514} I could see some kernel messages as well, earlier today: Jun 24 10:14:21 B-ras kernel: node: ID [20498d]: type 'tee', 0 hooks, flags 0x9, 0 refs, mpd1295-vlan340-44-lt: Jun 24 10:14:21 B-ras kernel: Last active @ /usr/src/sys/netgraph/ng_base.c, line 804 Jun 24 10:14:21 B-ras kernel: problem discovered at file /usr/src/sys/netgraph/ng_base.c, line 3709 Jun 24 10:14:21 B-ras kernel: KDB: stack backtrace: Jun 24 10:14:21 B-ras kernel: #0 0xffffffff80b1af87 at kdb_backtrace+0x67 Jun 24 10:14:21 B-ras kernel: #1 0xffffffff80c46671 at ng_send_fn1+0xc1 Jun 24 10:14:21 B-ras kernel: #2 0xffffffff80c3e9c4 at ng_destroy_hook+0x334 Jun 24 10:14:21 B-ras kernel: #3 0xffffffff80c4289b at ng_apply_item+0x3eb Jun 24 10:14:21 B-ras kernel: #4 0xffffffff80c48475 at ngthread+0x315 Jun 24 10:14:21 B-ras kernel: #5 0xffffffff80a88c55 at fork_exit+0x85 Jun 24 10:14:21 B-ras kernel: #6 0xffffffff80ec5f8e at fork_trampoline+0xe Jun 24 10:14:21 B-ras kernel: Accessing freed node: ID [20498d]: type 'tee'= , 0 hooks, flags 0x9, 1 refs, mpd1295-vlan340-44-lt: Jun 24 10:14:21 B-ras kernel: Last active @ /usr/src/sys/netgraph/ng_base.c, line 3709 Jun 24 10:14:21 B-ras kernel: problem discovered at file /usr/src/sys/netgraph/ng_base.c, line 2455 Jun 24 10:14:21 B-ras kernel: KDB: stack backtrace: Jun 24 10:14:21 B-ras kernel: #0 0xffffffff80b1af87 at kdb_backtrace+0x67 Jun 24 10:14:21 B-ras kernel: #1 0xffffffff80c42711 at ng_apply_item+0x261 Jun 24 10:14:21 B-ras kernel: #2 0xffffffff80c4217e at ng_snd_item+0x1ee Jun 24 10:14:21 B-ras kernel: #3 0xffffffff80c3e9c4 at ng_destroy_hook+0x334 Jun 24 10:14:21 B-ras kernel: #4 0xffffffff80c4289b at ng_apply_item+0x3eb Jun 24 10:14:21 B-ras kernel: #5 0xffffffff80c48475 at ngthread+0x315 Jun 24 10:14:21 B-ras kernel: #6 0xffffffff80a88c55 at fork_exit+0x85 Jun 24 10:14:21 B-ras kernel: #7 0xffffffff80ec5f8e at fork_trampoline+0xe Jun 24 10:14:21 B-ras kernel: Accessing freed node: ID [20498d]: type 'tee'= , 0 hooks, flags 0x9, 1 refs, mpd1295-vlan340-44-lt: Jun 24 10:14:21 B-ras kernel: Last active @ /usr/src/sys/netgraph/ng_base.c, line 2455 Jun 24 10:14:22 B-ras kernel: problem discovered at file /usr/src/sys/netgraph/ng_base.c, line 2335 Jun 24 10:14:22 B-ras kernel: KDB: stack backtrace: Jun 24 10:14:22 B-ras kernel: #0 0xffffffff80b1af87 at kdb_backtrace+0x67 Jun 24 10:14:22 B-ras kernel: #1 0xffffffff80c42308 at ng_snd_item+0x378 Jun 24 10:14:22 B-ras kernel: #2 0xffffffff80c3e9c4 at ng_destroy_hook+0x334 Jun 24 10:14:22 B-ras kernel: #3 0xffffffff80c4289b at ng_apply_item+0x3eb Jun 24 10:14:22 B-ras kernel: #4 0xffffffff80c48475 at ngthread+0x315 Jun 24 10:14:22 B-ras kernel: #5 0xffffffff80a88c55 at fork_exit+0x85 Jun 24 10:14:22 B-ras kernel: #6 0xffffffff80ec5f8e at fork_trampoline+0xe Jun 24 10:14:22 B-ras kernel: Accessing freed node: ID [20498d]: type 'tee'= , 0 hooks, flags 0x9, 0 refs, mpd1295-vlan340-44-lt: Jun 24 10:14:22 B-ras kernel: Last active @ /usr/src/sys/netgraph/ng_base.c, line 2335 Jun 24 10:14:22 B-ras kernel: problem discovered at file /usr/src/sys/netgraph/ng_base.c, line 804 Jun 24 10:14:22 B-ras kernel: KDB: stack backtrace: Jun 24 10:14:22 B-ras kernel: #0 0xffffffff80b1af87 at kdb_backtrace+0x67 Jun 24 10:14:22 B-ras kernel: #1 0xffffffff80c3ee46 at ng_unref_node+0x146 Jun 24 10:14:22 B-ras kernel: #2 0xffffffff80c42326 at ng_snd_item+0x396 Jun 24 10:14:22 B-ras kernel: #3 0xffffffff80c3e9c4 at ng_destroy_hook+0x334 Jun 24 10:14:22 B-ras kernel: #4 0xffffffff80c4289b at ng_apply_item+0x3eb Jun 24 10:14:22 B-ras kernel: #5 0xffffffff80c48475 at ngthread+0x315 Jun 24 10:14:22 B-ras kernel: #6 0xffffffff80a88c55 at fork_exit+0x85 Jun 24 10:14:22 B-ras kernel: #7 0xffffffff80ec5f8e at fork_trampoline+0xe Let me know if you need something else. Thanks. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 03:15:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68BBED968AE for ; Wed, 28 Jun 2017 03:15:56 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A72B2E3C for ; Wed, 28 Jun 2017 03:15:55 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (124-148-108-84.dyn.iinet.net.au [124.148.108.84]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v5S3FlEq089606 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 27 Jun 2017 20:15:51 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Sporadic TCP/RST sent to client To: Youssef GHORBAL , Matt Joras Cc: "freebsd-net@freebsd.org" , "nparhar@gmail.com" , "sthaug@nethelp.no" References: <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> <20170627.125426.74697078.sthaug@nethelp.no> From: Julian Elischer Message-ID: <94419886-55fe-20f2-3e01-4161b02dff80@freebsd.org> Date: Wed, 28 Jun 2017 11:15:42 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 03:15:56 -0000 On 28/6/17 2:31 am, Youssef GHORBAL wrote: > [...] > >> Further, I would argue that round robin is not a valid 802.3ad/802.1AX >> algorithm, per how it defines a frame distributor: >> >> "This standard does not mandate any particular distribution >> algorithm(s); however, any distribution algorithm shall ensure that, >> when frames are received by a Frame Collector as specified in 5.2.3, >> the algorithm shall not cause: >> a) Misordering of frames that are part of any given conversation, or >> b) Duplication of frames. >> >> The above requirement to maintain frame ordering is met by ensuring >> that all frames that compose a given conversation are transmitted on a >> single link in the order that they are generated by the MAC Client; >> hence, this requirement does not involve the addition (or >> modification) of any information to the MAC frame, nor any buffering >> or processing on the part of the corresponding Frame Collector in >> order to reorder frames." > I tend to agree with you on this point. > >>> Sure, I was just wondering if the FreeBSD network stack was built with the fact that each flow needs to arrive on the same NIC and the system was designed with this assumption in mind or not. >>> >>> I reported it here, thinking that maybe it's a subtle buggy corner case and maybe the community was interesting to know about and maybe fix : >>> >>> - If the stack is working as expected and was built with the assumption that each incoming flow needs to stick to a NIC during it's lifetime, maybe documentation needs to be more explicit regarding this situation. In that case I'll file documentation enhancement bug report. >>> - If the stack is misbehaving, maybe help the community identify the root cause and help fixing it >>> >> As far as I can tell, as Navdeep noted, there's no unexpected >> behaviour in your case. "Flows" are a concept that the protocols, in >> this case TCP, knows about. The devices themselves (Ethernet cards) >> usually have mechanics to make packet delivery decisions based on flow >> information (e.g. RSS hashing), but as far as I know that is generally >> limited within a single port, so it doesn't really help in the general >> case of a lagg. > So the fact that it works most of the time is just a "happy" coincidence. But it's not a behaviour to relay on. Right ? > > Anyway, thank you very much for your help and the clarification on this issue. Basically, while the packets are "on the wire" anything could happen. It would be theoretically possible for a lagg interface to do packet inspection and to not deliver (for some small amount of time) packets which showed sign of a missing earlier packet, but this is the job of other layers (e.g. tcp) and it would have negative impacts on performance of otherwise well behaved flows as well. It would have to actually look at tcp headers and that is WAY out of scope for lagg. !t would also need a lot of state to be stored if it were to hold state for all recent sessions, and make the code MUCH more complicated. Let tcp be tcp and push the linux owner to use something better. > > Youssef > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Wed Jun 28 04:31:05 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA019D97C8B for ; Wed, 28 Jun 2017 04:31:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA9476539E for ; Wed, 28 Jun 2017 04:31:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5S4UxWd048525 for ; Wed, 28 Jun 2017 04:31:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 04:31:00 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 04:31:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #84 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #83) It seems you hit another problem in the libc/stdio. Konstantin produced ano= ther patch for this problem: https://reviews.freebsd.org/file/data/nthhi3ogesbuhnjclgmu/PHID-FILE-ahlgny= gvinulibyezovs/D11246.diff ( https://reviews.freebsd.org/D11246 ) I'm very sorry not notifying you about this patch before. I ran my tests wi= th this patch applied too and have no hangs. Please apply it similarly and additionally to the "syslog" patch and restart mpd with web server enabled. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 12:35:48 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3684FDA182D for ; Wed, 28 Jun 2017 12:35:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE0C779EE for ; Wed, 28 Jun 2017 12:35:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SCZljn084514 for ; Wed, 28 Jun 2017 12:35:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 12:35:47 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 12:35:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #85 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #84) Hi Eugene, Don't worry, no problem. I tried to apply Konstantin patch but i had some rejects: # cat lib/libc/stdio/fgets.c.rej @@ -53,17 +53,17 @@ fgets(char * __restrict buf, int n, FILE * __restrict fp) { size_t len; - char *s; + char *s, *ret; unsigned char *p, *t; - FLOCKFILE(fp); + FLOCKFILE_CANCELSAFE(fp); ORIENT(fp, -1); if (n <=3D 0) { /* sanity check */ fp->_flags |=3D __SERR; errno =3D EINVAL; - FUNLOCKFILE(fp); - return (NULL); + ret =3D NULL; + goto end; } s =3D buf; @@ -76,8 +76,8 @@ if (__srefill(fp)) { /* EOF/error: stop with partial or no line = */ if (!__sfeof(fp) || s =3D=3D buf) { - FUNLOCKFILE(fp); - return (NULL); + ret =3D NULL; + goto end; } break; } # cat lib/libc/stdio/fgetwln.c.rej @@ -45,13 +45,14 @@ wchar_t * fgetwln_l(FILE * __restrict fp, size_t *lenp, locale_t locale) { + wchar_t *ret; wint_t wc; size_t len; int savserr; FIX_LOCALE(locale); - FLOCKFILE(fp); + FLOCKFILE_CANCELSAFE(fp); ORIENT(fp, 1); savserr =3D fp->_flags & __SERR; # cat lib/libc/stdio/fgetws.c.rej @@ -46,14 +46,14 @@ fgetws_l(wchar_t * __restrict ws, int n, FILE * __restrict fp, locale_t locale) { int sret; - wchar_t *wsp; + wchar_t *wsp, *ret; size_t nconv; const char *src; unsigned char *nl; FIX_LOCALE(locale); struct xlocale_ctype *l =3D XLOCALE_CTYPE(locale); - FLOCKFILE(fp); + FLOCKFILE_CANCELSAFE(fp); ORIENT(fp, 1); if (n <=3D 0) { @@ -113,12 +113,14 @@ goto error; ok: *wsp =3D L'\0'; - FUNLOCKFILE(fp); + ret =3D ws; +end: + FUNLOCKFILE_CANCELSAFE(); return (ws); error: - FUNLOCKFILE(fp); - return (NULL); + ret =3D NULL; + goto end; } wchar_t * Should i update to latest FreeBSD-11 STABLE? I'm using 11.0-p0. Thanks --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 13:51:34 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAE14DA2A9A for ; Wed, 28 Jun 2017 13:51:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA84379C6B for ; Wed, 28 Jun 2017 13:51:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SDpWRY074297 for ; Wed, 28 Jun 2017 13:51:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 13:51:33 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 13:51:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #86 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #85) Yes, that would be easiest thing. All these patches apply to 11-STABLE just fine. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 13:52:43 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A42F3DA2C23 for ; Wed, 28 Jun 2017 13:52:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 920C579DE7 for ; Wed, 28 Jun 2017 13:52:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SDqhM9081753 for ; Wed, 28 Jun 2017 13:52:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 13:52:43 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 13:52:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #87 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #85) And there is no need to apply "syslog" patch to 11-STABLE as it has already been merged. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 14:43:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6980DA39C5 for ; Wed, 28 Jun 2017 14:43:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A45C67BCDA for ; Wed, 28 Jun 2017 14:43:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SEhjec024475 for ; Wed, 28 Jun 2017 14:43:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 14:43:45 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 14:43:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #88 from Konstantin Belousov --- (In reply to Eugene Grosbein from comment #84) I very much doubt that the rtld lock leak (?) can be caused by stdio bugs. In other words, apply the debugging rtld patch I posted 06-16, and follow t= he accompanying instructions from there. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 14:48:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F367DA3BF5 for ; Wed, 28 Jun 2017 14:48:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C85B7BFEB for ; Wed, 28 Jun 2017 14:48:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SEm0G4030705 for ; Wed, 28 Jun 2017 14:48:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 14:48:01 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 14:48:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #89 from Cassiano Peixoto --- (In reply to Konstantin Belousov from comment #88) Hi Konstantin, rtld patch has been applied since them as i said on comment #67. Do you need some additional debug info from core file? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 20:22:43 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0625DA94CC for ; Wed, 28 Jun 2017 20:22:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E35A136C for ; Wed, 28 Jun 2017 20:22:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SKMcuM020816 for ; Wed, 28 Jun 2017 20:22:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 20:22:38 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 20:22:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #90 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #87) Eugene, Sorry my delay but i was updating to 11.1-BETA3 r320450M.=20 On updated i've applied the following patches: - libc/stdio from https://reviews.freebsd.org/D11246 - updated to mpd5.8_1 with web server enabled - ipfw patch - libthr patch (rtld) - in_mcast patch - ip_input patch - ng_iface patch - stf patch Let me know if you need anything else. Thanks --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jun 28 23:44:40 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4D1D86E46 for ; Wed, 28 Jun 2017 23:44:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2CCA66616 for ; Wed, 28 Jun 2017 23:44:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5SNidcg087711 for ; Wed, 28 Jun 2017 23:44:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 28 Jun 2017 23:44:40 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 23:44:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #91 from Konstantin Belousov --- (In reply to Cassiano Peixoto from comment #89) See comment #52 for instructions. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 00:11:18 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22E53D87EE0 for ; Thu, 29 Jun 2017 00:11:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 109DC67B1C for ; Thu, 29 Jun 2017 00:11:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5T0BHje086301 for ; Thu, 29 Jun 2017 00:11:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 00:11:17 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 00:11:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #92 from Cassiano Peixoto --- (In reply to Konstantin Belousov from comment #91) Did you read everything on my comment #83? Everything you asked on comment = #52 is there. Or am i missed something? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 01:29:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE9A1D8AAB1 for ; Thu, 29 Jun 2017 01:29:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DCC446E882 for ; Thu, 29 Jun 2017 01:29:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5T1T87Q092324 for ; Thu, 29 Jun 2017 01:29:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 95267] packet drops periodically appear Date: Thu, 29 Jun 2017 01:29:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jpaetzel@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 01:29:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D95267 Josh Paetzel changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Overcome By Events Status|In Progress |Closed CC| |jpaetzel@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 02:16:30 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68374D8D028 for ; Thu, 29 Jun 2017 02:16:30 +0000 (UTC) (envelope-from dreadiscool@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F39D6706E4 for ; Thu, 29 Jun 2017 02:16:29 +0000 (UTC) (envelope-from dreadiscool@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id 62so69166810wmw.1 for ; Wed, 28 Jun 2017 19:16:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Qoa0T7o4AkLNho2JIAVzg56jBabd7Q9WJPhnuVwiU+s=; b=PUDvYI+HH/ruuYio+Q1deZpiGCVR3e9EX0brI1aVfGQKVTDUHFrs+wcjU5s96QCSBw EI1v0lYga1tKg8Ffb9QsEYg3iQlavWZcJuxTxROZvgPY8dwKO+wAu/lerXDS25d6JTSf ENBLL6q3i8zm/x5eOxD+4FQXkRueQx2jL34ajTH6OUNkb8S+pq7Fq+is1VdfOjaat5F6 lb9VNZKes3Fu9/S6p6EEvbP0ZGwTjz39X6m73F2PSrEEMjH9jTvQ0F95uF0nfKScObCP 2x+fwc6aEkW6vxd4NpSxot8cdaAUJsOeu7ZbpARps1hjjxQQyxuDQeAYxy/GzMJ7dmuZ Y8dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Qoa0T7o4AkLNho2JIAVzg56jBabd7Q9WJPhnuVwiU+s=; b=QRUz6WsANfoIjrItfB8RZ4PghVtUcucWW5vJukhks3tOVmNzH9b/0TDSP764qI2wwC +y9Ldoj1fmgtnBHFQlwxp3PNHQ/QXm7jvh78ELUwWlARU06f5dds/wUj561Qosg0etoX 9G81pFdoJsJasUvl2jthIzLrI1twdcKGKcu+OiDQNXUExg44Y58Sdg6ZB6Yhu5bUiFjq l+JWJf1eoMnHBC+iV3793HRONLKNmgkTpdEK6oBY0m9a/CzrF6Tuz9KZUc2txhTI7N5g iSeXmC0kTlx8FK1tNEYM6eTjU05spALbKXKi9RoVej4939ycH3Y6SYQtY8jBxYC4spYj RCxw== X-Gm-Message-State: AKS2vOwglmpI2A3qG605+GTzCv8g5XiQG6t2J1CXqE2IzWxSCF351/ZR YTYkaBxers6N/pSnP7yZwx0r67ZXnLLV X-Received: by 10.28.236.83 with SMTP id k80mr9866814wmh.52.1498702588439; Wed, 28 Jun 2017 19:16:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.150.86 with HTTP; Wed, 28 Jun 2017 19:16:27 -0700 (PDT) From: Paras Jha Date: Wed, 28 Jun 2017 22:16:27 -0400 Message-ID: Subject: Netmap with bonded interfaces To: FreeBSD Net Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 02:16:30 -0000 Hi all, I have a bonded interface bond0 which enslaves eth4 and eth5. When trying to open the devices eth4 or eth5 via netmap, I get a "device in use" error. Opening the bond0 interface directly in netmap works, however it is in emulated mode (as expected of a pseudointerface) What is the idiomatic way to proceed in such a situation, without compromising on speed? Thanks From owner-freebsd-net@freebsd.org Thu Jun 29 04:39:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A7A6D92138 for ; Thu, 29 Jun 2017 04:39:11 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0DD89752F0 for ; Thu, 29 Jun 2017 04:39:11 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-wr0-x22e.google.com with SMTP id r103so183433950wrb.0 for ; Wed, 28 Jun 2017 21:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=5/WXo45hn4YKgTW1oEBAuUo6pIbenmTdiOMgjCHQrDQ=; b=Yt0hP3NPXEyvBaOtzw64Xhm/GVZYaenxhqEx2dceEVAW4tGMoZ5uKLPxIClehtXHfb tMzRREAJOfGPfF5vW5d/KN0ZeIwAVZahPLvrWf6e730lN+VKLlsdK+oDiQRbQG9dzNjd b+ePhBQ+/hfkW3u36jb9wKJ81qvSZ+o8WsZTmndRJeVDUbu+6SkhbC6WTLx/xpY7AoTI A3W38o8/LukVw+onumKxRUDkdzLPKf7wLtvD5sci8VgYInbL4SQ0cp3ycTsIByFjqQKs wFS9ujCk/gP0v0y3I6fVazZ+xK8BzwMb2wo35i+kYH4Jz5LdoVxhUDyQyqV+xfo784a6 rf7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=5/WXo45hn4YKgTW1oEBAuUo6pIbenmTdiOMgjCHQrDQ=; b=ZOfkGbP8njVATYG9sQfiQyPIEJBu2PI9YhBFRYTYGbMaVL2hDyRzXOt+LFaeI5Rm7X oWO0E3xGS15+hQORb56j9wqMQkkJ+/BLl0BfcBC4SSSRVHqaAvpD2/Ijk+xFeH9UPMV1 y/rY+IG7ABZFW34V+3s/uNvKu6+4lPPmu/UASqmSjfvz6uNQJqfpGZe1PFUslsGjlUV/ aIdaX48pjatqtn3ZzCCPbFq1GBrVXsQW5mVl/nkgPzdlDiggxrqEu/m5rwwg0uOU2mp7 DqHhIwifuhVeAoMvskK5OSQ83E3qQzR7cp5n+aFaLOydUyOLGkE93gc/S0LjX9OyTjBy 4FKw== X-Gm-Message-State: AKS2vOzo4o/MELFWn5bX/iXIz+Y9zINz4rw57ksxXZSLauhQOIfwVLhH Tc5CCp7SWynf3w5L0HIqfBfXnkB8WA== X-Received: by 10.223.161.219 with SMTP id v27mr21908432wrv.135.1498711149290; Wed, 28 Jun 2017 21:39:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.139.73 with HTTP; Wed, 28 Jun 2017 21:39:08 -0700 (PDT) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 29 Jun 2017 06:39:08 +0200 Message-ID: Subject: Re: Netmap with bonded interfaces To: Paras Jha Cc: FreeBSD Net Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 04:39:11 -0000 Hi, It is an expected behaviour that you cannot open eth4 and eth5 if they are bond, as the device are being used by the lagg pseudo-driver. Since this driver does not have netmap support for the moment being, there is no way you can get the native mode performance if you use lagg. If you just need some failover in your application, you could just implement a simple failover mechanism in your application (e.g. the application opens both netmap:eth4 and netmap:eth5, and decides which one to use for transmission depending on which one is up...). Cheers, Vincenzo 2017-06-29 4:16 GMT+02:00 Paras Jha : > Hi all, > > I have a bonded interface bond0 which enslaves eth4 and eth5. When trying > to open the devices eth4 or eth5 via netmap, I get a "device in use" error. > Opening the bond0 interface directly in netmap works, however it is in > emulated mode (as expected of a pseudointerface) > > What is the idiomatic way to proceed in such a situation, without > compromising on speed? > > Thanks > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione From owner-freebsd-net@freebsd.org Thu Jun 29 05:59:30 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CD09D939D9 for ; Thu, 29 Jun 2017 05:59:30 +0000 (UTC) (envelope-from dreadiscool@gmail.com) Received: from mail-wr0-x22d.google.com (mail-wr0-x22d.google.com [IPv6:2a00:1450:400c:c0c::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 83E787705D for ; Thu, 29 Jun 2017 05:59:29 +0000 (UTC) (envelope-from dreadiscool@gmail.com) Received: by mail-wr0-x22d.google.com with SMTP id r103so184005608wrb.0 for ; Wed, 28 Jun 2017 22:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=488xKzA9z0tFTPuTKN2sBj+T31V5CBlwMDgZn1v+UuE=; b=mErSbm25RpegmDa/+dEdpqQ9Pzm77vvOASIaEtnwd2Q1c1pSnQlyk0yh3ft/WwvxQ9 D34a438iNJlXbKAfh1Lp96kzs3LoRbUaO+X25w1Za6vgFtWvhqVCKtAXCZCOG6/6E07/ doimrOffYQo5vBWJmFWfpdk66dTKIN2DCocj3l7FIy/AksHUbsWg8Bdkf03Jf0P79B5t GbyDuCde3KkRTVYvwD+no3RjQqfdRD6BlesuNCu4Vzv6SzxR98AC6eUTETfN6mHwmBFI 0qtJ0stX+QX6GIWDyX30SwN6P/DiEqHsE7xK8vIIDIaCqFfwTY+V9JQJdKaiYgVKAWMs 5MbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=488xKzA9z0tFTPuTKN2sBj+T31V5CBlwMDgZn1v+UuE=; b=Yrwro0XTEwQ9UX5CJ6BFjnBQdJnFKif6g/XMoIoKQlB2Oz84d8NSQZM2lGOX/m3Ywj Rd3nO3i/yBIH01KQlRYJtN1jiE08Tmv556qCbYN7yyBfbFbx0JAbz9vObXX1JSjSdLo1 0EgmeUJFHrFkSoEJr8KQNez1OUNw7bK+vGHz4JSCXs9KdD7/rk4IPehUAFA3FsKrJKCQ 26MJfGFMtLW0Ljk1smQOivogPXfjTrEWvcqzzi1BLMyagDllUGqJo1Tp5AUnXUXLuF3x goeZ1FW2jW83FIzm4CD5gETlhxoQzvKWUXhuk9tzMPcUJ6hVHtOsj0NDbamwBvpFt28V 2m1A== X-Gm-Message-State: AKS2vOwWziqnYwIT4PoLv9nENq5qhW0GYyje4C/sJPHSHjwBnwPmkA+H xjngBjvwZ+MoutOvx68wLkrrH2/WOQ== X-Received: by 10.223.171.69 with SMTP id r5mr23008944wrc.57.1498715966758; Wed, 28 Jun 2017 22:59:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.150.86 with HTTP; Wed, 28 Jun 2017 22:59:26 -0700 (PDT) In-Reply-To: References: From: Paras Jha Date: Thu, 29 Jun 2017 01:59:26 -0400 Message-ID: Subject: Re: Netmap with bonded interfaces To: Vincenzo Maffione Cc: FreeBSD Net Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 05:59:30 -0000 It's possible to bypass this by unloading and reloading the patched network driver On Thu, Jun 29, 2017 at 12:39 AM, Vincenzo Maffione wrote: > Hi, > It is an expected behaviour that you cannot open eth4 and eth5 if they > are bond, as the device are being used by the lagg pseudo-driver. > Since this driver does not have netmap support for the moment being, there > is no way you can get the native mode performance if you use lagg. > > If you just need some failover in your application, you could just > implement a simple failover mechanism in your application (e.g. the > application opens both netmap:eth4 and netmap:eth5, and decides which one > to use for transmission depending on which one is up...). > > Cheers, > Vincenzo > > 2017-06-29 4:16 GMT+02:00 Paras Jha : > >> Hi all, >> >> I have a bonded interface bond0 which enslaves eth4 and eth5. When trying >> to open the devices eth4 or eth5 via netmap, I get a "device in use" >> error. >> Opening the bond0 interface directly in netmap works, however it is in >> emulated mode (as expected of a pseudointerface) >> >> What is the idiomatic way to proceed in such a situation, without >> compromising on speed? >> >> Thanks >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Vincenzo Maffione > From owner-freebsd-net@freebsd.org Thu Jun 29 07:54:45 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F534D95FAD for ; Thu, 29 Jun 2017 07:54:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D6707A1DD for ; Thu, 29 Jun 2017 07:54:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5T7sipj064303 for ; Thu, 29 Jun 2017 07:54:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 07:54:44 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 07:54:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #93 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #90) Make sure you have latest revision of https://reviews.freebsd.org/D11246 pa= tch as Konstantin updated it several hours ago to include "fseeko" chunk and yo= ur last hang and traces show a reference to that "fseeko". --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 09:28:53 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D0E3D97F01 for ; Thu, 29 Jun 2017 09:28:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A2C67CEF1 for ; Thu, 29 Jun 2017 09:28:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5T9SoJa051281 for ; Thu, 29 Jun 2017 09:28:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 09:28:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 09:28:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #94 from Cassiano Peixoto --- (In reply to Eugene Grosbein from comment #93) Ok i reverted the patch and applied the new one, but i got an error when compiling libc: cc -O2 -pipe -I/usr/src/lib/libc/include -I/usr/src/include -I/usr/src/lib/libc/amd64 -DNLS -D__DBINTERFACE_PRIVATE -I/usr/src/contrib/gdtoa -I/usr/src/contrib/libc-vis -DINET6 -I/usr/obj/usr/src/lib/libc -I/usr/src/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/usr/src/lib/libmd -I/usr/src/contrib/jemalloc/include -I/usr/src/contrib/tzcode/stdtime -I/usr/src/lib/libc/stdtime -I/usr/src/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/usr/src/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING -DSYMBOL_VERSIONING= -MD -MF.depend.stdio.o -MTstdio.o -std=3Dgnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments=20 -I/usr/src/lib/libutil -I/usr/src/lib/msun/amd64 -I/usr/src/lib/msun/x86 -I/usr/src/lib/msun/src -c /usr/src/lib/libc/stdio/stdio.c -o stdio.o /usr/src/lib/libc/stdio/stdio.c:179:1: error: redefinition of '__stdio_cancel_cleanup' __stdio_cancel_cleanup(void * arg) ^ /usr/src/lib/libc/stdio/stdio.c:171:1: note: previous definition is here __stdio_cancel_cleanup(void * arg) ^ 1 error generated. *** Error code 1 Stop. make: stopped in /usr/src/lib/libc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 10:57:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77EDCD99D6E for ; Thu, 29 Jun 2017 10:57:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65D847FDE2 for ; Thu, 29 Jun 2017 10:57:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TAv2fB088702 for ; Thu, 29 Jun 2017 10:57:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 10:57:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 10:57:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #95 from Eugene Grosbein --- (In reply to Cassiano Peixoto from comment #94) It seems your source tree is broken now. Try to clean is up and apply all patches again: cd /usr/src && svnlite cleanup --remove-unversioned && svnlite revert -R . --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 11:08:38 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68F79D9A250 for ; Thu, 29 Jun 2017 11:08:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5713780356 for ; Thu, 29 Jun 2017 11:08:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TB8bKb034835 for ; Thu, 29 Jun 2017 11:08:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 11:08:38 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 11:08:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #96 from Cassiano Peixoto --- (In reply to Cassiano Peixoto from comment #94) Let it go, something messed when applied the patch. I re-synced the src and applied fine. So, patched applied and mpd5 recompiled. Let's watch. Thanks. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 15:13:58 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BC8BD9F7C2 for ; Thu, 29 Jun 2017 15:13:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08A4C3689 for ; Thu, 29 Jun 2017 15:13:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TFDqln096348 for ; Thu, 29 Jun 2017 15:13:57 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 15:13:52 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 15:13:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #97 from Konstantin Belousov --- (In reply to Cassiano Peixoto from comment #92) Indeed, everything I asked for, is in the comment #83. Now I cannot make a sense from the reported line 4802 of rtld.c in the backtrace for the thread 100704. There is no lock call neither in stable/11 nor in HEAD, at this li= ne.=20 What exact version of the sources do you use, from which branch ? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 15:44:30 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D474D9FE85 for ; Thu, 29 Jun 2017 15:44:30 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-wr0-x230.google.com (mail-wr0-x230.google.com [IPv6:2a00:1450:400c:c0c::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC0E064550 for ; Thu, 29 Jun 2017 15:44:29 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-wr0-x230.google.com with SMTP id k67so191041743wrc.2 for ; Thu, 29 Jun 2017 08:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1iPhzckTMplWzCRUzNmPwpVlS0o7u2Sj1e0RENU9e4E=; b=HceIvM/fvO1JpOSqefi+BynNLt2MYOOxdZ1ZC9iFB5yfLZqNYM06uRXHwNQIhtR+nu x1FkQzElJHXuWu5/HGyqGnOJRxbL5gFo5NZDK8b/HgJ8+YCJZ1fsUORRp64oeQo3J2C3 5H7yVTMhmjCxiQBOVkNFWnpreLtWalDp+b+cPjCr8LJ+QH9vq2cQamZIVvJfe3HRyQEy vBNC1SbgZZSbk8VYgj7ZlOhqD43rqc2w2WUwIxJbvwgY2YoFTGBtUa5qGOzUQDsz1Usa CtXmbbN42nPV5Tj7++i2S/FKzYNXxMfc+O8Lfw4SbHVMs2lkyxsJB8/xjcJQirz+2er6 IR1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1iPhzckTMplWzCRUzNmPwpVlS0o7u2Sj1e0RENU9e4E=; b=Quw4+8YoPTJhwaSB8pMyNG7limzZyg6Sdcv2jGedeFAw8vYRpv4n1Q038omKmXzzIF uWXiT/t3Z17UibhL0gMZDnzo2cJNrrrXY1MTCVkqor9ML+CO+o47/sYpuBe+15AT4d3l rjeGLLu5TO4ZqWOKUMMLi+DB93wpedJIPtmL1opBOfDhU3UUiITcSoIRtNvlk8PW1NX4 VG0F2iawK2cmpqIQEMPODa3ar5aOZL2Dt++rnI3SNZRAo0athVOdcV5MSPJrYkHYkRt8 0AiqtOlAlRYvInpVgEND75y6RixiuX+84ru9MZvB21SnGnYf0A8MKA5Fi6kSjU5tQ/Fy snSg== X-Gm-Message-State: AKS2vOwx+4begxZR0W/Z6nPQvNJHpMCRmvNp9z3EOG44q3X1+8HtO95z zZ/ZzybEtjkPsiLIB09nydyg2/dz7g== X-Received: by 10.223.171.69 with SMTP id r5mr24666405wrc.57.1498751067917; Thu, 29 Jun 2017 08:44:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.49.132 with HTTP; Thu, 29 Jun 2017 08:44:27 -0700 (PDT) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 29 Jun 2017 08:44:27 -0700 Message-ID: Subject: Re: Netmap with bonded interfaces To: Paras Jha Cc: FreeBSD Net Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 15:44:30 -0000 Bypass what? 2017-06-28 22:59 GMT-07:00 Paras Jha : > It's possible to bypass this by unloading and reloading the patched > network driver > > On Thu, Jun 29, 2017 at 12:39 AM, Vincenzo Maffione > wrote: > >> Hi, >> It is an expected behaviour that you cannot open eth4 and eth5 if they >> are bond, as the device are being used by the lagg pseudo-driver. >> Since this driver does not have netmap support for the moment being, >> there is no way you can get the native mode performance if you use lagg. >> >> If you just need some failover in your application, you could just >> implement a simple failover mechanism in your application (e.g. the >> application opens both netmap:eth4 and netmap:eth5, and decides which one >> to use for transmission depending on which one is up...). >> >> Cheers, >> Vincenzo >> >> 2017-06-29 4:16 GMT+02:00 Paras Jha : >> >>> Hi all, >>> >>> I have a bonded interface bond0 which enslaves eth4 and eth5. When trying >>> to open the devices eth4 or eth5 via netmap, I get a "device in use" >>> error. >>> Opening the bond0 interface directly in netmap works, however it is in >>> emulated mode (as expected of a pseudointerface) >>> >>> What is the idiomatic way to proceed in such a situation, without >>> compromising on speed? >>> >>> Thanks >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> >> >> -- >> Vincenzo Maffione >> > > -- Vincenzo Maffione From owner-freebsd-net@freebsd.org Thu Jun 29 16:56:18 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BA90DA105D for ; Thu, 29 Jun 2017 16:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3952066692 for ; Thu, 29 Jun 2017 16:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TGuDkP035264 for ; Thu, 29 Jun 2017 16:56:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 16:56:13 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 16:56:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #98 from Eugene Grosbein --- (In reply to Konstantin Belousov from comment #97) That's from releng/11.0: https://svnweb.freebsd.org/base/releng/11.0/libexec/rtld-elf/rtld.c?annotat= e=3D304456#l4802 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 17:00:23 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DB74DA115B for ; Thu, 29 Jun 2017 17:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B770668AE for ; Thu, 29 Jun 2017 17:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TH0LRc045522 for ; Thu, 29 Jun 2017 17:00:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 17:00:21 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 17:00:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #99 from Cassiano Peixoto --- (In reply to Konstantin Belousov from comment #97) When it happend i was using 11.0-release-p0. But yesterday i updated to 11.1-beta3. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 18:36:47 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA917DA2C72 for ; Thu, 29 Jun 2017 18:36:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A84086E308 for ; Thu, 29 Jun 2017 18:36:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TIahxR039449 for ; Thu, 29 Jun 2017 18:36:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Thu, 29 Jun 2017 18:36:43 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 18:36:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #100 from Konstantin Belousov --- (In reply to Cassiano Peixoto from comment #99) Uh, ok. So lets wait for the issue to re-appear on latest stable. There w= ere some fixes that might be relevant. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 18:53:58 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0A70DA3245 for ; Thu, 29 Jun 2017 18:53:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE5946EE7D for ; Thu, 29 Jun 2017 18:53:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TIrv5d082225 for ; Thu, 29 Jun 2017 18:53:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Thu, 29 Jun 2017 18:53:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 18:53:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |regression Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 18:54:19 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65F75DA32D7 for ; Thu, 29 Jun 2017 18:54:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5211D6EF77 for ; Thu, 29 Jun 2017 18:54:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TIsJXq082772 for ; Thu, 29 Jun 2017 18:54:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220351] Path MTU discovery works only after resetting an internal interface MTU (mpd5, ng_nat, ipfw) Date: Thu, 29 Jun 2017 18:54:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 18:54:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220351 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 18:58:24 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 492C7DA3721 for ; Thu, 29 Jun 2017 18:58:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 377606F4AD for ; Thu, 29 Jun 2017 18:58:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TIwN84088588 for ; Thu, 29 Jun 2017 18:58:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220217] deadlock on enc and pf Date: Thu, 29 Jun 2017 18:58:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 18:58:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220217 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 19:53:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFC3CDA4A6E for ; Thu, 29 Jun 2017 19:53:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CA3D372051 for ; Thu, 29 Jun 2017 19:53:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TJrEBP037116 for ; Thu, 29 Jun 2017 19:53:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220351] Path MTU discovery works only after resetting an internal interface MTU (mpd5, ng_nat, ipfw) Date: Thu, 29 Jun 2017 19:53:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 19:53:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220351 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |eugen@freebsd.org Status|New |Open CC| |eugen@freebsd.org --- Comment #2 from Eugene Grosbein --- (In reply to Martin Birgmeier from comment #1) Lowering interface MTU of internal machine also lowers its default route 'm= tu' attribute, see output of the command 'route -n get default'. Raising interface MTU back does NOT restore 'mtu' attribute of the default route, so the kernel still uses 1492 as maximum IP packet size for packets going via this route. Correct solution for your problem is 'set iface enable tcpmssfix' command in the mpd.conf, see /usr/local/etc/mpd5/mpd.conf.sample for example. This way= you can keep default MTU=3D1500 for internal hosts and have no MTU-related netw= ork problems. Please test and report back. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jun 29 20:29:53 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C501DA507B for ; Thu, 29 Jun 2017 20:29:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE65072C4C for ; Thu, 29 Jun 2017 20:29:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5TKTqWW038186 for ; Thu, 29 Jun 2017 20:29:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Thu, 29 Jun 2017 20:29:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hselasky@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jun 2017 20:29:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 Hans Petter Selasky changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hselasky@FreeBSD.org --- Comment #1 from Hans Petter Selasky --- Hi, Are you using RSS? There hasn`t been any LRO related changes recently, so the crash is likely = in code outside the LRO code. --HPS --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 07:45:59 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B485DD8AD28 for ; Fri, 30 Jun 2017 07:45:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A31F483FEC for ; Fri, 30 Jun 2017 07:45:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5U7jxbK046599 for ; Fri, 30 Jun 2017 07:45:59 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 07:45:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: iz-rpi03@hs-karlsruhe.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 07:45:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #2 from iz-rpi03@hs-karlsruhe.de --- (In reply to Hans Petter Selasky from comment #1) Hi, not knowingly. I had to look up what RSS means. https://wiki.freebsd.org/NetworkRSS does not mention an Intel(R) PRO/1000 network interface (em), so I thing I do not use it. I am trying to find the change which causes this, but as the machine is an older laptopt building new kernels take its time.=20 Ralf --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 07:57:05 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 278F5D8B15B for ; Fri, 30 Jun 2017 07:57:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AA538460A for ; Fri, 30 Jun 2017 07:57:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5U7v48A071485 for ; Fri, 30 Jun 2017 07:57:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220217] deadlock on enc and pf Date: Fri, 30 Jun 2017 07:57:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 07:57:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220217 --- Comment #7 from Kristof Provost --- (In reply to Andrey V. Elsukov from comment #5) Thanks! I missed the inpcb being passed through netpfil. That explains why = it usually works. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 08:11:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74569D8B733 for ; Fri, 30 Jun 2017 08:11:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62DE484BE4 for ; Fri, 30 Jun 2017 08:11:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5U8B7cJ029147 for ; Fri, 30 Jun 2017 08:11:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 08:11:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hselasky@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 08:11:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #3 from Hans Petter Selasky --- RSS means "options RSS" in the kernel config Second question: Are you using hyperthreading? Can you try to enter: machdep.hyperthreading_allowed=3D0 in /boot/loader.conf Is this issue reproducible? --HPS --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 09:28:40 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5F72D8CE6F for ; Fri, 30 Jun 2017 09:28:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C70F221F4 for ; Fri, 30 Jun 2017 09:28:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5U9SeHF053624 for ; Fri, 30 Jun 2017 09:28:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 09:28:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: iz-rpi03@hs-karlsruhe.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 09:28:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #4 from iz-rpi03@hs-karlsruhe.de --- There is no "option RSS" in the used kernel config. So, the answer is: No. Hyperthreading was used. As you suggested I disabled it via /boot/loader.conf, but the panic happend again. It changed its cause to "page fault while in kernel mode" but almost not the place. "tcp_lro_flush" instead of the former "tcp_lro_flush_all". Ralf Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 01 fault virtual address =3D 0x55ea51aa fault code =3D supervisor write, page not present instruction pointer =3D 0x20:0xc7f3f21b stack pointer =3D 0x28:0xe37d97bc frame pointer =3D 0x28:0xe37d97e8 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D resume, IOPL =3D 0 current process =3D 0 (if_io_tqg_1) trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1498813503 KDB: stack backtrace: #0 0xc07dadaf at kdb_backtrace+0x4f #1 0xc079ccb3 at vpanic+0x133 #2 0xc079cb7b at panic+0x1b #3 0xc0ae38fe at trap_fatal+0x31e #4 0xc0ae3943 at trap_pfault+0x33 #5 0xc0ae304e at trap+0x2be #6 0xc0ad1fea at calltrap+0x6 #7 0xc096bb4f at tcp_do_segment+0x219f #8 0xc0968d67 at tcp_input+0x13a7 #9 0xc08f39a6 at ip_input+0x256 #10 0xc089328c at netisr_dispatch_src+0xcc#13 0xc087e65b at ether_nh_input+0x35b #14 0xc089328c at netisr_dispatch_src+0xcc #15 0xc0893550 at netisr_dispatch+0x20 #16 0xc087dc3a at ether_input+0x2a #17 0xc096dfc5 at tcp_lro_flush+0x1d5 Uptime: 6m23s Physical memory: 3523 MB Dumping 149 MB: 134 118 102 86 70 54 38 22 6 Reading symbols from /boot/kernel.r320396.crash/snd_hda.ko...Reading symbols from /usr/lib/debug//boot/kernel.r320396.crash/snd_hda.ko.debug...done. done. Reading symbols from /boot/kernel.r320396.crash/sound.ko...Reading symbols = from /usr/lib/debug//boot/kernel.r320396.crash/sound.ko.debug...done. done. Reading symbols from /boot/kernel.r320396.crash/cuse.ko...Reading symbols f= rom /usr/lib/debug//boot/kernel.r320396.crash/cuse.ko.debug...done. done. Reading symbols from /boot/kernel.r320396.crash/ums.ko...Reading symbols fr= om /usr/lib/debug//boot/kernel.r320396.crash/ums.ko.debug...done. done. __curthread () at ./machine/pcpu.h:225 225 __asm("movl %%fs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:225 #1 doadump (textdump=3D-949457280) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xc079c924 in kern_reboot (howto=3D) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xc079cceb in vpanic (fmt=3D, ap=3D) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xc079cb7b in panic (fmt=3D0xc0b23936 "%s") at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xc0ae38fe in trap_fatal (frame=3D, eva=3D) at /usr/src/sys/i386/i386/trap.c:978 #6 0xc0ae3943 in trap_pfault (frame=3D,=20 usermode=3D, eva=3D) at /usr/src/sys/i386/i386/trap.c:804 #7 0xc0ae304e in trap (frame=3D) at /usr/src/sys/i386/i386/trap.c:512 #8 #9 0xc7f3f21b in ?? () #10 0xc096bb4f in tcp_do_segment (m=3D, th=3D= ,=20 so=3D, tp=3D, drop_hdrlen=3D,=20 tlen=3D, iptos=3D,=20 ti_locked=3D) at /usr/src/sys/netinet/tcp_input.c:2444 #11 0xc0968d67 in tcp_input (mp=3D, offp=3D,= =20 proto=3D) at /usr/src/sys/netinet/tcp_input.c:1191 #12 0xc08f39a6 in ip_input (m=3D0x0) at /usr/src/sys/netinet/ip_input.c:823 #13 0xc089328c in netisr_dispatch_src (proto=3D,=20 source=3D, m=3D0xc7f3f219) at /usr/src/sys/net/netisr.c:= 1120 #14 0xc0893550 in netisr_dispatch (proto=3D1, m=3D0xc8172000) at /usr/src/sys/net/netisr.c:1211 #15 0xc087d9b0 in ether_demux (ifp=3D0xc77ca800, m=3D0x0) at /usr/src/sys/net/if_ethersubr.c:848 #16 0xc087e65b in ether_input_internal (ifp=3D0xc77ca800, m=3D0xc7f3f219) at /usr/src/sys/net/if_ethersubr.c:637 #17 ether_nh_input (m=3D) at /usr/src/sys/net/if_ethersubr.c= :667 #18 0xc089328c in netisr_dispatch_src (proto=3D,=20 source=3D, m=3D0xc7f3f219) at /usr/src/sys/net/netisr.c:= 1120 #19 0xc0893550 in netisr_dispatch (proto=3D5, m=3D0xc8172000) at /usr/src/sys/net/netisr.c:1211 #20 0xc087dc3a in ether_input (ifp=3D0xc77ca800, m=3D0x0) at /usr/src/sys/net/if_ethersubr.c:757 #21 0xc096dfc5 in tcp_lro_flush (lc=3D0xc77ad424, le=3D) at /usr/src/sys/netinet/tcp_lro.c:394 #22 0xc096e161 in tcp_lro_rx_done (lc=3D0xc77ad424) at /usr/src/sys/netinet/tcp_lro.c:284 #23 tcp_lro_flush_all (lc=3D) at /usr/src/sys/netinet/tcp_lro.c:532 #24 0xc088dc90 in iflib_rxeof (budget=3D16, rxq=3D) at /usr/src/sys/net/iflib.c:2564 #25 _task_fn_rx (context=3D) at /usr/src/sys/net/iflib.c:3499 #26 0xc07d9aa8 in gtaskqueue_run_locked (queue=3D0xc7688000) at /usr/src/sys/kern/subr_gtaskqueue.c:329 #27 0xc07d97c7 in gtaskqueue_thread_loop (arg=3D0xc7671814) at /usr/src/sys/kern/subr_gtaskqueue.c:504 #28 0xc0764a16 in fork_exit (callout=3D0xc07d9720 ,= =20 arg=3D, frame=3D) at /usr/src/sys/kern/kern_fork.c:1038 #29 (kgdb)=20 #11 0xc0893550 at netisr_dispatch+0x20 #12 0xc087d9b0 at ether_demux+0x140 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 10:45:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7DCBD8E180 for ; Fri, 30 Jun 2017 10:45:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6394641E4 for ; Fri, 30 Jun 2017 10:45:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5UAjEvW055685 for ; Fri, 30 Jun 2017 10:45:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 10:45:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hselasky@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 10:45:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #5 from Hans Petter Selasky --- Adding Sean Bruno. I also notice your hardware is 32-bit. Have you seen this issue with 64-bit kernels? --HPS --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 11:39:29 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D9B6D8F0B1 for ; Fri, 30 Jun 2017 11:39:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C11265639 for ; Fri, 30 Jun 2017 11:39:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5UBdTcp000743 for ; Fri, 30 Jun 2017 11:39:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 11:39:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: iz-rpi03@hs-karlsruhe.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 11:39:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #6 from iz-rpi03@hs-karlsruhe.de --- > I also notice your hardware is 32-bit. Have you seen this issue with 64-b= it kernels? No, but on this hardware I have not run a 64-bit kernel yet. On a different hardware a very current 64-bit kernel with also an "em" interface does not show this issue. Ralf --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 12:19:27 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA6FBD905DA for ; Fri, 30 Jun 2017 12:19:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7BDA66B66 for ; Fri, 30 Jun 2017 12:19:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5UCJRR4036342 for ; Fri, 30 Jun 2017 12:19:27 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 12:19:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: iz-rpi03@hs-karlsruhe.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 12:19:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #7 from iz-rpi03@hs-karlsruhe.de --- Just an intermediate result: A r320008 32-bit kernel, no hyperthreading, panics as well with "privileged instruction fault" in "tcp_lro_flush_all". Ralf --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 15:46:43 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58984D94637 for ; Fri, 30 Jun 2017 15:46:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46DCD728D6 for ; Fri, 30 Jun 2017 15:46:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v5UFkhNX030672 for ; Fri, 30 Jun 2017 15:46:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Fri, 30 Jun 2017 15:46:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: oleg.nauman@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 15:46:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 oleg.nauman@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |oleg.nauman@gmail.com --- Comment #8 from oleg.nauman@gmail.com --- I'm observing crashes too ( CURRENT/i386 r320466 ), for example crash due to incoming SSH connection attempt: __curthread () at ./machine/pcpu.h:225 225 __asm("movl %%fs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:225 #1 doadump (textdump=3D-968634112) at ../../../kern/kern_shutdown.c:318 #2 0xc06e8954 in kern_reboot (howto=3D) at ../../../kern/kern_shutdown.c:386 #3 0xc06e8ceb in vpanic (fmt=3D, ap=3D0xea5c56ec "K\336\235\300H\325\065\306\001") at ../../../kern/kern_shutdown.c:779 #4 0xc06e8bab in panic (fmt=3D0xc092e2de "%s") at ../../../kern/kern_shutdown.c:710 #5 0xc08eee71 in trap_fatal (frame=3D0xea5c584c, eva=3D) at ../../../i386/i386/trap.c:978 #6 0xc08eefbb in trap_pfault (frame=3D0xea5c584c, usermode=3D0, eva=3D) at ../../../i386/i386/trap.c:890 #7 0xc08ee5de in trap (frame=3D) at ../../../i386/i386/trap.c:512 #8 #9 0xc6be0a1b in ?? () #10 0xc082ef73 in tcp_do_segment (m=3D, th=3D, so=3D, tp=3D, drop_hdrlen=3D, tlen=3D, iptos=3D, ti_locked=3D) at ../../../netinet/tcp_input.c:2444 #11 0xc082c3a1 in tcp_input (mp=3D, offp=3D, proto=3D) at ../../../netinet/tcp_input.c:1191 #12 0xc0820a98 in ip_input (m=3D0x0) at ../../../netinet/ip_input.c:823 #13 0xc07d57db in netisr_dispatch_src (proto=3D, source=3D, m=3D0xc6be0a18) at ../../../net/netisr.c:1120 #14 0xc07d5aa0 in netisr_dispatch (proto=3D1, m=3D0xc6c01800) at ../../../net/netisr.c:1211 #15 0xc07c74b2 in ether_demux (ifp=3D0xc634e800, m=3D0x0) at ../../../net/if_ethersubr.c:848 #16 0xc07c8140 in ether_input_internal (ifp=3D0xc634e800, m=3D0xc6be0a18) at ../../../net/if_ethersubr.c:637 #17 ether_nh_input (m=3D) at ../../../net/if_ethersubr.c:667 #18 0xc07d57db in netisr_dispatch_src (proto=3D, source=3D, m=3D0xc6be0a18) at ../../../net/netisr.c:1120 #19 0xc07d5aa0 in netisr_dispatch (proto=3D5, m=3D0xc6c01800) at ../../../net/netisr.c:1211 #20 0xc07c773a in ether_input (ifp=3D0xc634e800, m=3D0x0) at ../../../net/if_ethersubr.c:757 #21 0xc04f5058 in age_rxeof (sc=3D, rxrd=3D) at ../../../dev/age/if_age.c:2442 #22 age_rxintr (rr_prod=3D4, count=3D, sc=3D) at ../../../dev/age/if_age.c:2488 #23 age_int_task (arg=3D, pending=3D1) at ../../../dev/age/if_age.c:2167 #24 0xc0735bfc in taskqueue_run_locked (queue=3D0xc631a300) at ../../../kern/subr_taskqueue.c:454 #25 0xc0736ae7 in taskqueue_thread_loop (arg=3D0xc6344a6c) at ../../../kern/subr_taskqueue.c:746 #26 0xc06b8b06 in fork_exit (callout=3D0xc0736a40 , arg=3D, frame=3D) at ../../../kern/kern_fork.c:1038 #27 (kgdb) machdep.hyperthreading_allowed is set to 0 My system also reproducible panics on named reconfigure/flush/shutdown even= ts with ( partially ) similar backtrace , as well as panics caused by IPC ; for example: __curthread () at ./machine/pcpu.h:225 225 __asm("movl %%fs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:225 #1 doadump (textdump=3D-968633856) at ../../../kern/kern_shutdown.c:318 #2 0xc06e88c4 in kern_reboot (howto=3D) at ../../../kern/kern_shutdown.c:386 #3 0xc06e8c5b in vpanic (fmt=3D, ap=3D0xefd5c73c "\340\334\235\300\310\370\266\306\001") at ../../../kern/kern_shutdown.c:779 #4 0xc06e8b1b in panic (fmt=3D0xc092e18e "%s") at ../../../kern/kern_shutdown.c:710 #5 0xc08eed21 in trap_fatal (frame=3D0xefd5c878, eva=3D) at ../../../i386/i386/trap.c:978 #6 0xc08eea38 in trap (frame=3D) at ../../../i386/i386/trap.c:704 #7 #8 0xc6bcda1b in ?? () #9 0xc0770281 in unp_connect2 (so=3D, so2=3D, req=3D) at ../../../kern/uipc_usrreq.c:1497 #10 0xc076ff17 in unp_connectat (fd=3D, so=3D, nam=3D, td=3D) at ../../../kern/uipc_usrreq.c:1446 #11 0xc076d510 in unp_connect (so=3D0xc71c9400, nam=3D0xc662d500, td=3D) at ../../../kern/uipc_usrreq.c:1310 #12 uipc_connect (so=3D0xc71c9400, nam=3D0xc662d500, td=3D) at ../../../kern/uipc_usrreq.c:587 #13 0xc076a042 in kern_connectat (td=3D, dirfd=3D-100, fd=3D, sa=3D0xc662d500) at ../../../kern/uipc_syscalls.c= :505 #14 0xc0769f49 in sys_connect (td=3D0xc6bcda18, uap=3D0xc6b6f988) at ../../../kern/uipc_syscalls.c:470 #15 0xc08ef679 in syscallenter (td=3D) at ../../../i386/i386/../../kern/subr_syscall.c:132 #16 syscall (frame=3D) at ../../../i386/i386/trap.c:1103 #17 #18 0x283a4747 in ?? () Backtrace stopped: Cannot access memory at address 0xbfbfe794 (kgdb) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jun 30 15:51:44 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E5F1D949DF for ; Fri, 30 Jun 2017 15:51:44 +0000 (UTC) (envelope-from kfodil-lemelin@xiplink.com) Received: from smtp73.iad3a.emailsrvr.com (smtp73.iad3a.emailsrvr.com [173.203.187.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D31772F3F for ; Fri, 30 Jun 2017 15:51:44 +0000 (UTC) (envelope-from kfodil-lemelin@xiplink.com) Received: from smtp34.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by smtp34.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 5E35B24C86; Fri, 30 Jun 2017 11:42:06 -0400 (EDT) X-Auth-ID: kfodil-lemelin@xiplink.com Received: by smtp34.relay.iad3a.emailsrvr.com (Authenticated sender: kfodil-lemelin-AT-xiplink.com) with ESMTPSA id 3F67C24D8A; Fri, 30 Jun 2017 11:42:06 -0400 (EDT) X-Sender-Id: kfodil-lemelin@xiplink.com Received: from [10.10.1.167] ([UNAVAILABLE]. [192.252.130.194]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA) by 0.0.0.0:465 (trex/5.7.12); Fri, 30 Jun 2017 11:42:06 -0400 To: FreeBSD Net From: Karim Fodil-Lemelin Subject: m_move_pkthdr leaves m_nextpkt 'dangling' Message-ID: <59567148.1020902@xiplink.com> Date: Fri, 30 Jun 2017 11:42:00 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2017 15:51:44 -0000 Hi, As many of you know, when dealing with IP fragments the kernel will build a list of packets (fragments) chained together through the m_nextpkt pointer. This is all good until someone tries to do a M_PREPEND on one of the packet in the chain and the M_PREPEND has to create an extra mbuf to prepend at the beginning of the chain. When doing so m_move_pkthdr is called to copy the current PKTHDR fields (tags and flags) to the mbuf that was prepended. The function also does: to->m_pkthdr = from->m_pkthdr; This, for the case I am interested in, essentially leaves the 'from' mbuf with a dangling pointer m_nextpkt pointing to the next fragment. While this is mostly harmless because only mbufs of pkthdr types are supposed to have m_nextpkt it triggers some panics when running with INVARIANTS in NetGraph (see ng_base.c :: CHECK_DATA_MBUF(m)): ... if (n->m_nextpkt != NULL) \ panic("%s: m_nextpkt", __func__); \ } ... So I would like to propose the following patch: @@ -442,10 +442,11 @@ m_move_pkthdr(struct mbuf *to, struct mbuf *from) if ((to->m_flags & M_EXT) == 0) to->m_data = to->m_pktdat; to->m_pkthdr = from->m_pkthdr; /* especially tags */ SLIST_INIT(&from->m_pkthdr.tags); /* purge tags from src */ from->m_flags &= ~M_PKTHDR; + from->m_nextpkt = NULL; } It will reset the m_nextpkt so we don't have two mbufs pointing to the same next packet. This is fairly harmless and solves a problem for us here at XipLink. Best regards, Karim. From owner-freebsd-net@freebsd.org Sat Jul 1 16:48:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57C24D8E05B for ; Sat, 1 Jul 2017 16:48:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46B6267633 for ; Sat, 1 Jul 2017 16:48:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v61Gm8RO067055 for ; Sat, 1 Jul 2017 16:48:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220358] panic in tcp_lro_flush_all Date: Sat, 01 Jul 2017 16:48:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hselasky@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jul 2017 16:48:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220358 --- Comment #9 from Hans Petter Selasky --- If this issue is a recent regression and the issue easily reproduces, can y= ou try to bisect, I.E. binary search the exact revision which is causing this issue. Most easily this can be done when using GIT (See git bisect) --=20 You are receiving this mail because: You are the assignee for the bug.=