From owner-freebsd-performance@FreeBSD.ORG Sun May 24 15:21:02 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CBCEF71 for ; Sun, 24 May 2015 15:21:02 +0000 (UTC) Received: from green.field.hu (green.field.hu [217.20.130.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 196531BE4 for ; Sun, 24 May 2015 15:21:01 +0000 (UTC) Received: from green.field.hu (localhost [127.0.0.1]) by green.field.hu (Postfix) with ESMTP id 5E3EF250A64 for ; Sun, 24 May 2015 17:12:58 +0200 (CEST) X-Virus-Scanned: by Amavisd-new at field.hu Received: from green.field.hu ([127.0.0.1]) by green.field.hu (green.field.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W4NxqI909vcv for ; Sun, 24 May 2015 17:12:55 +0200 (CEST) Received: from [192.168.52.11] (1F2EC6DD.catv.pool.telekom.hu [31.46.198.221]) by green.field.hu (Postfix) with ESMTPA id 6488F250A4F for ; Sun, 24 May 2015 17:12:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=field.hu; s=mail; t=1432480375; bh=QwT3LKZLzmNAtV21B+g2BuMW/gA489DDMSzpff17lbY=; h=Date:From:To:Subject; b=G0DCAMttxMImPl6XhAXRIbXO2G/IBGskUwJ4SGC5BRFNWcg7hXvz04bZP8rzgSmEX KPm9iYdFNzFn/xy7wBAw/JM9BBIG/PubN6IV8ZIWONr0bo1UgODq9IFadcHIeC+qWG 4UCESt59uQo+98/L9Rr88xmE7nGySXOok/BU7nqE= Message-ID: <5561EA73.20005@field.hu> Date: Sun, 24 May 2015 17:12:51 +0200 From: Cs User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: FreeBSD 10.1-REL - network unaccessible after high traffic Content-Type: text/plain; charset=iso-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 May 2015 15:21:02 -0000 Hi all, I have two FreeBSd 10.1-RELEASE servers connected to each other. They were connected via cross link, but they are connected to a cisco switch now. When transferring huge files (50-500GB backup files) via Gigabit (it is important!) the network randomly dies. The backup runs every day/week and sometimes the connection is ok for months sometimes it happens twice a week. When the network dies I can log in to the server via IPMI and use the console everything is OK, but can't send anything out on the network. ifconfig em0 down/up doesn't help nor netif restart. The problem never occured when I used 100Mbit connection between them, but it was 3com NIC (xl), gigabit adapter is Intel (em0). When I limit the transfer rate (rsync bandwith limit or ipfw pipe) the problem is much more rare. I tried to set these tuning parameters on both servers with different buffer size but nothing helped: # cat /etc/sysctl.conf security.bsd.see_other_uids=0 net.inet.tcp.recvspace=512000 net.route.netisr_maxqlen=2048 kern.ipc.nmbclusters=1310720 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 kern.ipc.soacceptqueue=32768 # cat /boot/loader.conf geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ipfw_load="YES" net.inet.ip.fw.default_to_accept=1 kern.maxusers=4096 accf_data_load="YES" Any ideas? Thanks guys! From owner-freebsd-performance@FreeBSD.ORG Mon May 25 02:31:14 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94D6ACCC for ; Mon, 25 May 2015 02:31:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C307D27 for ; Mon, 25 May 2015 02:31:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from jre-mbp.elischer.org (ppp121-45-225-35.lns20.per1.internode.on.net [121.45.225.35]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t4P2V5x3059224 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 24 May 2015 19:31:08 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <55628963.9080805@freebsd.org> Date: Mon, 25 May 2015 10:30:59 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Cs , freebsd-performance@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic References: <5561EA73.20005@field.hu> In-Reply-To: <5561EA73.20005@field.hu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 02:31:14 -0000 On 5/24/15 11:12 PM, Cs wrote: > Hi all, > > I have two FreeBSd 10.1-RELEASE servers connected to each other. > They were connected via cross link, but they are connected to a > cisco switch now. When transferring huge files (50-500GB backup > files) via Gigabit (it is important!) the network randomly dies. The > backup runs every day/week and sometimes the connection is ok for > months sometimes it happens twice a week. When the network dies I > can log in to the server via IPMI and use the console everything is > OK, but can't send anything out on the network. ifconfig em0 down/up > doesn't help nor netif restart. The problem never occured when I > used 100Mbit connection between them, but it was 3com NIC (xl), > gigabit adapter is Intel (em0). When I limit the transfer rate > (rsync bandwith limit or ipfw pipe) the problem is much more rare. did you have the problem with no switch? is he duplex setting correct? > > I tried to set these tuning parameters on both servers with > different buffer size but nothing helped: > > # cat /etc/sysctl.conf > security.bsd.see_other_uids=0 > net.inet.tcp.recvspace=512000 > net.route.netisr_maxqlen=2048 > kern.ipc.nmbclusters=1310720 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.recvbuf_max=16777216 > kern.ipc.soacceptqueue=32768 > > # cat /boot/loader.conf > geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) > ipfw_load="YES" > net.inet.ip.fw.default_to_accept=1 > kern.maxusers=4096 > accf_data_load="YES" > > Any ideas? Thanks guys! > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Mon May 25 06:34:51 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1E29F64 for ; Mon, 25 May 2015 06:34:50 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A533C59 for ; Mon, 25 May 2015 06:34:49 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (localhost [127.0.0.1]) by green.field.hu (Postfix) with ESMTP id 9E9EC250AA5 for ; Mon, 25 May 2015 08:34:46 +0200 (CEST) X-Virus-Scanned: by Amavisd-new at field.hu Received: from green.field.hu ([127.0.0.1]) by green.field.hu (green.field.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gZktSxdxEHd8 for ; Mon, 25 May 2015 08:34:39 +0200 (CEST) Received: from [192.168.52.11] (1F2EC6DD.catv.pool.telekom.hu [31.46.198.221]) by green.field.hu (Postfix) with ESMTPA id 2D55C250A61 for ; Mon, 25 May 2015 08:34:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=field.hu; s=mail; t=1432535679; bh=WGCStl3XBgvSHuAXkns7ujb+KCPTioJS9nFu/jUo3ts=; h=Date:From:To:Subject:References:In-Reply-To; b=nNhir/4HF+UhHnoWuJ+Cph6SIY0Rs0GFWbeiWqlGIaj6IquefWkmg/X8GWsYD3ZmW QveuVtlxXDUjVUo5XZibs2FCwzOVL4jWsxSJOX6UdnjHKiraPB1F8D4IMoKXdMdo6z Be1tKOO8zu7nBf8phx/VGuLOBIsPcETFf0oeq0+w= Message-ID: <5562C27A.5000604@field.hu> Date: Mon, 25 May 2015 08:34:34 +0200 From: Cs User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic References: <5561EA73.20005@field.hu> <55628963.9080805@freebsd.org> In-Reply-To: <55628963.9080805@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 06:34:51 -0000 Hi Julian, Yes, the problem was the same when I used cross link for two years. The duplex settings are identical on both servers. Server A: em1: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:24:52:66 inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Server B: em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:30:48:dd:fe:3e inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active I always suspected the 'em' driver and thought "it will be fixed in the next release", but after ~3 years I think I need to dig deep to find the root cause. Regards, Csaba 2015.05.25. 4:30 keltezéssel, Julian Elischer írta: > On 5/24/15 11:12 PM, Cs wrote: >> Hi all, >> >> I have two FreeBSd 10.1-RELEASE servers connected to each other. They >> were connected via cross link, but they are connected to a cisco >> switch now. When transferring huge files (50-500GB backup files) via >> Gigabit (it is important!) the network randomly dies. The backup runs >> every day/week and sometimes the connection is ok for months >> sometimes it happens twice a week. When the network dies I can log in >> to the server via IPMI and use the console everything is OK, but >> can't send anything out on the network. ifconfig em0 down/up doesn't >> help nor netif restart. The problem never occured when I used 100Mbit >> connection between them, but it was 3com NIC (xl), gigabit adapter is >> Intel (em0). When I limit the transfer rate (rsync bandwith limit or >> ipfw pipe) the problem is much more rare. > > did you have the problem with no switch? > is he duplex setting correct? > >> >> I tried to set these tuning parameters on both servers with different >> buffer size but nothing helped: >> >> # cat /etc/sysctl.conf >> security.bsd.see_other_uids=0 >> net.inet.tcp.recvspace=512000 >> net.route.netisr_maxqlen=2048 >> kern.ipc.nmbclusters=1310720 >> net.inet.tcp.sendbuf_max=16777216 >> net.inet.tcp.recvbuf_max=16777216 >> kern.ipc.soacceptqueue=32768 >> >> # cat /boot/loader.conf >> geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) >> ipfw_load="YES" >> net.inet.ip.fw.default_to_accept=1 >> kern.maxusers=4096 >> accf_data_load="YES" >> >> Any ideas? Thanks guys! >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Mon May 25 06:54:26 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18B27188 for ; Mon, 25 May 2015 06:54:26 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4A2EDE for ; Mon, 25 May 2015 06:54:25 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (localhost [127.0.0.1]) by green.field.hu (Postfix) with ESMTP id 0307C250A64 for ; Mon, 25 May 2015 08:54:24 +0200 (CEST) X-Virus-Scanned: by Amavisd-new at field.hu Received: from green.field.hu ([127.0.0.1]) by green.field.hu (green.field.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 18mEHYnoD67c for ; Mon, 25 May 2015 08:54:20 +0200 (CEST) Received: from [192.168.52.11] (1F2EC6DD.catv.pool.telekom.hu [31.46.198.221]) by green.field.hu (Postfix) with ESMTPA id 972CB250A6D for ; Mon, 25 May 2015 08:54:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=field.hu; s=mail; t=1432536860; bh=ZE+WyLm+hpblpw7gMMeR6Tc+k8kzXbT6HyniJ4C4Zgs=; h=Date:From:To:Subject:References:In-Reply-To; b=LbiqkaszAejqXQlZaHN2mjmnGYG7oWQHwT7hGKD7PB/XORES8TV5Wjxq69DmkcBwf rn/5J3iAwguk4/BEglYrXmsRsQAAQbJmHvVi1Bgw8Ec3adnng/EfYOm24pHwWwQL+n G4vDcvym7D4x7tuOn23VY94Hf7fjJsicpKqN9zqA= Message-ID: <5562C718.6040005@field.hu> Date: Mon, 25 May 2015 08:54:16 +0200 From: Cs User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic References: <5561EA73.20005@field.hu> <55628963.9080805@freebsd.org> <5562C27A.5000604@field.hu> In-Reply-To: <5562C27A.5000604@field.hu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 06:54:26 -0000 Hi All, Julian gave me an idea to increase MTU to 9000 but I'm not sure if it helps. Anyway I increased it on both servers. @Julian: the loader.conf and sysctl tuning parameters are ok? Regards, Csaba 2015.05.25. 8:34 keltezéssel, Cs írta: > Hi Julian, > > Yes, the problem was the same when I used cross link for two years. > The duplex settings are identical on both servers. > > Server A: > em1: flags=8843 metric 0 mtu 1500 > options=4219b > > ether 00:25:90:24:52:66 > inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > Server B: > em0: flags=8843 metric 0 mtu 1500 > options=4219b > > ether 00:30:48:dd:fe:3e > inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > I always suspected the 'em' driver and thought "it will be fixed in > the next release", but after ~3 years I think I need to dig deep to > find the root cause. > > Regards, > Csaba > > 2015.05.25. 4:30 keltezéssel, Julian Elischer írta: >> On 5/24/15 11:12 PM, Cs wrote: >>> Hi all, >>> >>> I have two FreeBSd 10.1-RELEASE servers connected to each other. >>> They were connected via cross link, but they are connected to a >>> cisco switch now. When transferring huge files (50-500GB backup >>> files) via Gigabit (it is important!) the network randomly dies. The >>> backup runs every day/week and sometimes the connection is ok for >>> months sometimes it happens twice a week. When the network dies I >>> can log in to the server via IPMI and use the console everything is >>> OK, but can't send anything out on the network. ifconfig em0 down/up >>> doesn't help nor netif restart. The problem never occured when I >>> used 100Mbit connection between them, but it was 3com NIC (xl), >>> gigabit adapter is Intel (em0). When I limit the transfer rate >>> (rsync bandwith limit or ipfw pipe) the problem is much more rare. >> >> did you have the problem with no switch? >> is he duplex setting correct? >> >>> >>> I tried to set these tuning parameters on both servers with >>> different buffer size but nothing helped: >>> >>> # cat /etc/sysctl.conf >>> security.bsd.see_other_uids=0 >>> net.inet.tcp.recvspace=512000 >>> net.route.netisr_maxqlen=2048 >>> kern.ipc.nmbclusters=1310720 >>> net.inet.tcp.sendbuf_max=16777216 >>> net.inet.tcp.recvbuf_max=16777216 >>> kern.ipc.soacceptqueue=32768 >>> >>> # cat /boot/loader.conf >>> geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) >>> ipfw_load="YES" >>> net.inet.ip.fw.default_to_accept=1 >>> kern.maxusers=4096 >>> accf_data_load="YES" >>> >>> Any ideas? Thanks guys! >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to >>> "freebsd-performance-unsubscribe@freebsd.org" >>> >> >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Mon May 25 07:25:26 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 644AC595 for ; Mon, 25 May 2015 07:25:26 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 457A9A3E for ; Mon, 25 May 2015 07:25:25 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from jre-mbp.elischer.org (ppp121-45-225-35.lns20.per1.internode.on.net [121.45.225.35]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t4P7PHUL060189 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 25 May 2015 00:25:20 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5562CE57.6070000@freebsd.org> Date: Mon, 25 May 2015 15:25:11 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Cs , freebsd-performance@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic References: <5561EA73.20005@field.hu> <55628963.9080805@freebsd.org> <5562C27A.5000604@field.hu> <5562C718.6040005@field.hu> In-Reply-To: <5562C718.6040005@field.hu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 07:25:26 -0000 On 5/25/15 2:54 PM, Cs wrote: > Hi All, > > Julian gave me an idea to increase MTU to 9000 but I'm not sure if > it helps. Anyway I increased it on both servers. > @Julian: the loader.conf and sysctl tuning parameters are ok? better mailing list might be -net. I don't see any obvious problems. It may be worth trying to turn off all the advanced features like TSO etc. > > Regards, > Csaba > > 2015.05.25. 8:34 keltezéssel, Cs írta: >> Hi Julian, >> >> Yes, the problem was the same when I used cross link for two years. >> The duplex settings are identical on both servers. >> >> Server A: >> em1: flags=8843 metric 0 >> mtu 1500 >> options=4219b >> >> ether 00:25:90:24:52:66 >> inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x >> nd6 options=29 >> media: Ethernet autoselect (1000baseT ) >> status: active >> >> Server B: >> em0: flags=8843 metric 0 >> mtu 1500 >> options=4219b >> >> ether 00:30:48:dd:fe:3e >> inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x >> nd6 options=29 >> media: Ethernet autoselect (1000baseT ) >> status: active >> >> I always suspected the 'em' driver and thought "it will be fixed in >> the next release", but after ~3 years I think I need to dig deep to >> find the root cause. >> >> Regards, >> Csaba >> >> 2015.05.25. 4:30 keltezéssel, Julian Elischer írta: >>> On 5/24/15 11:12 PM, Cs wrote: >>>> Hi all, >>>> >>>> I have two FreeBSd 10.1-RELEASE servers connected to each other. >>>> They were connected via cross link, but they are connected to a >>>> cisco switch now. When transferring huge files (50-500GB backup >>>> files) via Gigabit (it is important!) the network randomly dies. >>>> The backup runs every day/week and sometimes the connection is ok >>>> for months sometimes it happens twice a week. When the network >>>> dies I can log in to the server via IPMI and use the console >>>> everything is OK, but can't send anything out on the network. >>>> ifconfig em0 down/up doesn't help nor netif restart. The problem >>>> never occured when I used 100Mbit connection between them, but it >>>> was 3com NIC (xl), gigabit adapter is Intel (em0). When I limit >>>> the transfer rate (rsync bandwith limit or ipfw pipe) the problem >>>> is much more rare. >>> >>> did you have the problem with no switch? >>> is he duplex setting correct? >>> >>>> >>>> I tried to set these tuning parameters on both servers with >>>> different buffer size but nothing helped: >>>> >>>> # cat /etc/sysctl.conf >>>> security.bsd.see_other_uids=0 >>>> net.inet.tcp.recvspace=512000 >>>> net.route.netisr_maxqlen=2048 >>>> kern.ipc.nmbclusters=1310720 >>>> net.inet.tcp.sendbuf_max=16777216 >>>> net.inet.tcp.recvbuf_max=16777216 >>>> kern.ipc.soacceptqueue=32768 >>>> >>>> # cat /boot/loader.conf >>>> geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) >>>> ipfw_load="YES" >>>> net.inet.ip.fw.default_to_accept=1 >>>> kern.maxusers=4096 >>>> accf_data_load="YES" >>>> >>>> Any ideas? Thanks guys! >>>> _______________________________________________ >>>> freebsd-performance@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>> To unsubscribe, send any mail to >>>> "freebsd-performance-unsubscribe@freebsd.org" >>>> >>> >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to >>> "freebsd-performance-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" > > > From owner-freebsd-performance@FreeBSD.ORG Mon May 25 07:28:45 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AAEF769D; Mon, 25 May 2015 07:28:45 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62DE1A95; Mon, 25 May 2015 07:28:45 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (localhost [127.0.0.1]) by green.field.hu (Postfix) with ESMTP id 7E687250A73; Mon, 25 May 2015 09:28:43 +0200 (CEST) X-Virus-Scanned: by Amavisd-new at field.hu Received: from green.field.hu ([127.0.0.1]) by green.field.hu (green.field.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MQeFRLanhup7; Mon, 25 May 2015 09:28:39 +0200 (CEST) Received: from [192.168.52.11] (1F2EC6DD.catv.pool.telekom.hu [31.46.198.221]) by green.field.hu (Postfix) with ESMTPA id 2EEBE250A74; Mon, 25 May 2015 09:28:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=field.hu; s=mail; t=1432538919; bh=BwNUSM09DcthcYs/i4l6Awf+drKIkJ5sB+OrTnhB3fM=; h=Date:From:To:Subject:References:In-Reply-To; b=beXNUD0WquZkSSqxVBX48+8SR9cdderiKGSIiD/BfPykRSC2hJVGF8DN/4jJnY2OU q9DweoEp1BLuDWHa50RK4E0SqVHwS/piVwrn8+YhEb2ve9YI+EbG7lOSQrFJQoHitS L0D/1AYA6AWgTh3pL8B+tiMKKzY91QGbBcmDy9Yg= Message-ID: <5562CF22.60306@field.hu> Date: Mon, 25 May 2015 09:28:34 +0200 From: Cs User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Julian Elischer , freebsd-performance@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic References: <5561EA73.20005@field.hu> <55628963.9080805@freebsd.org> <5562C27A.5000604@field.hu> <5562C718.6040005@field.hu> <5562CE57.6070000@freebsd.org> In-Reply-To: <5562CE57.6070000@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 07:28:45 -0000 Thanks, will try that! 2015.05.25. 9:25 keltezéssel, Julian Elischer írta: > On 5/25/15 2:54 PM, Cs wrote: >> Hi All, >> >> Julian gave me an idea to increase MTU to 9000 but I'm not sure if it >> helps. Anyway I increased it on both servers. >> @Julian: the loader.conf and sysctl tuning parameters are ok? > > better mailing list might be -net. > > I don't see any obvious problems. > It may be worth trying to turn off all the advanced features like > TSO etc. > > >> >> Regards, >> Csaba >> >> 2015.05.25. 8:34 keltezéssel, Cs írta: >>> Hi Julian, >>> >>> Yes, the problem was the same when I used cross link for two years. >>> The duplex settings are identical on both servers. >>> >>> Server A: >>> em1: flags=8843 metric 0 mtu >>> 1500 >>> options=4219b >>> >>> ether 00:25:90:24:52:66 >>> inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x >>> nd6 options=29 >>> media: Ethernet autoselect (1000baseT ) >>> status: active >>> >>> Server B: >>> em0: flags=8843 metric 0 mtu >>> 1500 >>> options=4219b >>> >>> ether 00:30:48:dd:fe:3e >>> inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x >>> nd6 options=29 >>> media: Ethernet autoselect (1000baseT ) >>> status: active >>> >>> I always suspected the 'em' driver and thought "it will be fixed in >>> the next release", but after ~3 years I think I need to dig deep to >>> find the root cause. >>> >>> Regards, >>> Csaba >>> >>> 2015.05.25. 4:30 keltezéssel, Julian Elischer írta: >>>> On 5/24/15 11:12 PM, Cs wrote: >>>>> Hi all, >>>>> >>>>> I have two FreeBSd 10.1-RELEASE servers connected to each other. >>>>> They were connected via cross link, but they are connected to a >>>>> cisco switch now. When transferring huge files (50-500GB backup >>>>> files) via Gigabit (it is important!) the network randomly dies. >>>>> The backup runs every day/week and sometimes the connection is ok >>>>> for months sometimes it happens twice a week. When the network >>>>> dies I can log in to the server via IPMI and use the console >>>>> everything is OK, but can't send anything out on the network. >>>>> ifconfig em0 down/up doesn't help nor netif restart. The problem >>>>> never occured when I used 100Mbit connection between them, but it >>>>> was 3com NIC (xl), gigabit adapter is Intel (em0). When I limit >>>>> the transfer rate (rsync bandwith limit or ipfw pipe) the problem >>>>> is much more rare. >>>> >>>> did you have the problem with no switch? >>>> is he duplex setting correct? >>>> >>>>> >>>>> I tried to set these tuning parameters on both servers with >>>>> different buffer size but nothing helped: >>>>> >>>>> # cat /etc/sysctl.conf >>>>> security.bsd.see_other_uids=0 >>>>> net.inet.tcp.recvspace=512000 >>>>> net.route.netisr_maxqlen=2048 >>>>> kern.ipc.nmbclusters=1310720 >>>>> net.inet.tcp.sendbuf_max=16777216 >>>>> net.inet.tcp.recvbuf_max=16777216 >>>>> kern.ipc.soacceptqueue=32768 >>>>> >>>>> # cat /boot/loader.conf >>>>> geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) >>>>> ipfw_load="YES" >>>>> net.inet.ip.fw.default_to_accept=1 >>>>> kern.maxusers=4096 >>>>> accf_data_load="YES" >>>>> >>>>> Any ideas? Thanks guys! >>>>> _______________________________________________ >>>>> freebsd-performance@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>> To unsubscribe, send any mail to >>>>> "freebsd-performance-unsubscribe@freebsd.org" >>>>> >>>> >>>> _______________________________________________ >>>> freebsd-performance@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>> To unsubscribe, send any mail to >>>> "freebsd-performance-unsubscribe@freebsd.org" >>> >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to >>> "freebsd-performance-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" >> >> >> >