From owner-freebsd-net@FreeBSD.ORG Sun Jun 30 19:41:17 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3E4FCFBA; Sun, 30 Jun 2013 19:41:17 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pb0-x232.google.com (mail-pb0-x232.google.com [IPv6:2607:f8b0:400e:c01::232]) by mx1.freebsd.org (Postfix) with ESMTP id 1648F1991; Sun, 30 Jun 2013 19:41:17 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id wz7so4054274pbc.9 for ; Sun, 30 Jun 2013 12:41:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=NQfzf95BFZbIkJ1WhFqFhLtMXraSUcisEhHbj3vje2U=; b=Ge56S0IeVtg6XZHkW1idOKTG9D83dnDItMWeVOyGXGprAFz4N6jZqwgvByL9L2Ttw1 chxym8mMfa1U6ULglZupYEInifKZYqsWphXQPclSNxy2ezuZXqC0gaoFDsUFFuq+gzkA d3F3zGU0ZITu2nWy2tTQHI5Vc3zIVBrNf+TwzwJdhcAP4GSj1BnzBs27RDCbMHe0QBlk 2I2NzvFOU/FOaYgxOlAJU2CcZSyAU2uUK19KyMomkiR2xBtbkc1DR1Intle5b0eSor8P H96EcVBkJzqrN3Jf2jXc4EYs+iLYPK7dOYUQ1Rm/3UuAMJw0HDdUxqlSOi2EKS42aBVP PwGg== X-Received: by 10.68.14.6 with SMTP id l6mr20751193pbc.202.1372621276770; Sun, 30 Jun 2013 12:41:16 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id tq8sm18494277pbc.30.2013.06.30.12.41.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 30 Jun 2013 12:41:15 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <51D089D9.6080901@FreeBSD.org> Date: Sun, 30 Jun 2013 12:41:13 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130627 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: cxgbetool & hw filtering issues References: <51D03FCE.1060102@FreeBSD.org> In-Reply-To: <51D03FCE.1060102@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jun 2013 19:41:17 -0000 On 06/30/13 07:25, Alexander V. Chernikov wrote: > Hello list! > > While experimenting with Chelsio T440-CR (cxgbe) internal firewall, I'm > getting some kind of unexpected results: One bit of general advice to begin with: add "hitcnts 1" to all your filter rules and then you can see how many incoming packets hit that filter in the output of "cxgbetool t4nex0 filter list". I really should make hitcnts=1 the default in the driver. > filtering 'type ipv4 action drop' permits IPv4 TCP traffic with bad > checksum. It may be that a bad checksum makes it an invalid IPv4 packet to the chip and so it doesn't hit the "type ipv4" rule. There is an entirely separate knob available to have the chip drop bad packets if you don't want to see them. The default is to let them through so that users can examine them with tcpdump etc. > filtering 'type IPv6 action drop' permits IPv6 traffic to multicast > addresses (MLDv2, etc..) The DMAC is an L2 multicast address? Try "proto 58 hitcnts 1 action drop" to get these ICMP6 packets. > filtering 'ethtype 34525 action drop' (drop all IPv6) results in > 'CHELSIO_T4_SET_FILTER: Argument list too long' despite to what is said > in budget table from cxgbetool.8 This _would_ have gotten everything with ethertype ipv6 but the default filter mode doesn't have ethtype enabled, which is why it's complaining: # cxgbetool t4nex0 filter mode ipv4 ipv6 sip dip sport dport matchtype proto ivlan iport fcoe > filtering 'matchtype 4 action drop' or similar (4,5,4:0,4:4, 5:0, 5:5) > does not match anything despite some traffic definitely falls into that > conditions. > filtering 'action drop' and 'iport X action drop' filters IPv4 traffic > only. Strange. I use "iport X action drop hitcnts 1" as a packet black hole all the time. Were these the only filters when you tried them? Are you sure your packets didn't hit some other rule and were delivered as a result of that? Check the order in "cxgbetool t4nex0 filter list" Also, are you going by the ifnet rx stats as displayed by netstat etc.? Right now the driver fills the ifnet stats directly from hardware registers rather than counting the packets that it actually received from the chip. The hardware registers include packets that would have been delivered to the driver if no filters were present but are dropped due to a filter. > filter 'type ipv6 ...' can be set on (0,4,8,12,...) filter numbers > yelling 'CHELSIO_T4_SET_FILTER: Invalid argument' on other numbers. Yes, IPv6 filters take 4 tid's (non-IPv6 take 1) and these tid's have to start at a naturally aligned boundary. No way around this. > What can I do to debug further/fix this behavior? > > Some more questions: > Does anybody known how I can get/set total number of HW firewall > records? There is such tunable in Linux version. I will add a simple sysctl for this. For now you can indirectly figure this out from the output of "sysctl -n dev.t4nex.0.misc.tids" -- the FTIDs are the filter tids. For example I see 1456 filters on this card: trantor:~# sysctl -n dev.t4nex.0.misc.tids ATID range: 0-8191, in use: 0 TID range: 2048-18431, in use: 0 STID range: 0-511, in use: 0 FTID range: 512-1967 HW TID usage: 0 IP users, 0 IPv6 users trantor:~# echo $((1967 - 512 + 1)) 1456 > Is there any way to retrieve _host_ interface statistic (e.g. how much > traffic in packets/bytes are thrown to NIC driver)? cxgbe(4) doesn't count this stuff itself. Currently it just reads the hardware registers once per second and it's done. Software stats would have to be per queue (and then aggregated from time to time). I'll wait to see where the PCPU counter work in the kernel goes before reworking this part of the driver. Regards, Navdeep