From owner-freebsd-net@freebsd.org Mon Aug 1 14:16:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70DB9BAAFD3 for ; Mon, 1 Aug 2016 14:16:36 +0000 (UTC) (envelope-from robbelics@gmail.com) Received: from mail-yw0-x230.google.com (mail-yw0-x230.google.com [IPv6:2607:f8b0:4002:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C66C1D55 for ; Mon, 1 Aug 2016 14:16:36 +0000 (UTC) (envelope-from robbelics@gmail.com) Received: by mail-yw0-x230.google.com with SMTP id u134so173546933ywg.3 for ; Mon, 01 Aug 2016 07:16:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=nIY0H9b+QSZAQ2ZXi0FMKjvln+QY5lZn2+E063kg4NU=; b=dMzafd9kA0u6kluea35SyZZJYpf0qfR9WcUgd1BewaqPhUNaaqx3udrJwcW/DA8s+p qLjvNq6zeVEDolQXXc9EUKnlWvWqqDELGaspfEHMttp7UqTThvFC9zniCooMMKPfpETv dGWHf8vPwOafuax6XJjbMS4ODmN8CjiBbq2yBUJvNPVdI/pzWOeQRAaq+2Lf7vmInWUc SkCvzklWYvhHxD6umGbrYhh3KWN9WaY4zbrlw4rd9Zne4gU3U4J108OeKmfdUgV7NndR hM2QHf6X1RY3falqcjfwEczk3GLRxRtRLrnsYzVfknGDgKvv3rxpyL7/jI//AbZC3Pyb 4F1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=nIY0H9b+QSZAQ2ZXi0FMKjvln+QY5lZn2+E063kg4NU=; b=PriRIcJc7/Ylre5gM3cTXhnhRjmNIU/3N9gVocEZbze4Tl2A8rXMsVXngwo/8xTsam wXXSasp0n4fRgvES+NftMl3LtGGBxZBo24L86PE0PYOCla6hoLYG+tILZ0PCSKaJOc4x u4cUI7qHY20OOmA++4Sbj6q1SxrnfgVFky+X5A0xZvUvlyhy5eQ1ncSjggtv7h/NIRht Z/BaBxOfMT0dUcg5DMrUbmKzua2wlr+rY0IgKj5mU6plfnfLAN3fjTyfDLt4Wu+L97xx wiyrKoTH7tYv1c0EnZL3j7xNIG9h1ONXVQzR+4lhg196YpdA744u+VuVV2ciimjwOFQp V5oA== X-Gm-Message-State: AEkoouucTWi52EMwCX+Er9QjhCZzKVfJrg7r31Z8m96OoNyPUBCKM5V2MLctqQj0nXP9H1Rj4RXzk+eAmp9T/g== X-Received: by 10.13.213.3 with SMTP id x3mr45644493ywd.253.1470060995162; Mon, 01 Aug 2016 07:16:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.220.17 with HTTP; Mon, 1 Aug 2016 07:16:34 -0700 (PDT) From: Rob Belics Date: Mon, 1 Aug 2016 09:16:34 -0500 Message-ID: Subject: Troubleshooting Idrops To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2016 14:16:36 -0000 I have a VPS server running FreeBSD 10.3-RELEASE-p4 and nginx. It contains three very low volume web sites that have been up for about three years. I was tinkering with TLS and SSL ciphers by eliminating TLSv1 and TLSv1.1 with different ciphers when I noticed my daily "Network interface status" report one morning saying I was getting Idrops of 48665. Network interface status: Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop em0 1500 xx:xx:3c:cd:7e:c7 225569570 0 48665 923463 0 0 0 em0 - xxxx::xxx:3cf fe80::216:3cff:fe 0 - - 4 - - - em0 - 107.xxx.xx.xx mysite1.co 94833 - - 0 - - - em0 - 107.xxx.xx.0 mysite2.co 479981 - - 920067 - - - lo0 16384 783 0 0 783 0 0 0 lo0 - ::1 ::1 0 - - 0 - - - lo0 - xxxx::1%lo0 xxxx::1%lo0 0 - - 0 - - - lo0 - your-net localhost 783 - - 783 - - - I reverted my TLS/SSL changes but, the next day, that exact same number of Idrops happened and continued for a couple of days afterwards. I just don't know what I could have done to cause this and am looking for troubleshooting help since it's been so long since I've had to deal with this and forgotten nearly everything. All the sites seem to function normally and I should note that, besides the nginx server, there are also two nodejs servers listening via proxy. I do nothing with IPv6. Here is part of vmstat -z where I notice FAILs: ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Hash: 128, 0, 5, 26, 7, 0, 0 4 Bucket: 16, 0, 8, 496, 21344, 0, 0 6 Bucket: 24, 0, 0, 336, 121, 0, 0 8 Bucket: 32, 0, 2, 376, 1600, 0, 0 12 Bucket: 48, 0, 0, 0, 97831, 0, 0 16 Bucket: 64, 0, 12, 303, 9585, 8, 0 32 Bucket: 128, 0, 14, 389, 46423, 0, 0 64 Bucket: 256, 0, 20, 235, 48362, 0, 0 128 Bucket: 512, 0, 19, 101, 23133, 0, 0 mbuf_packet: 256, 30975, 256, 253,455561904,97330, 0 mbuf: 256, 30975, 2, 254, 2124523, 0, 0 mbuf_cluster: 2048, 4840, 509, 3, 17874,194660, 2 mbuf_jumbo_page: 4096, 2419, 0, 2, 10659, 0, 0 mbuf_jumbo_9k: 9216, 716, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 403, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 256 Bucket: 1024, 0, 27, 45, 56839,5856, 0 vmem btag: 28, 0, 6648, 1848, 68924, 58, 0 I ran netstat -s and can post that here if it's OK for something that big or if someone wants something specific from that. In addition, what can I do to see these drops without having to wait till the next day for the report? I know I can do netstat -i but that contains drops for the current day.