Date: Fri, 25 Apr 2014 11:37:11 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= <gerrit.kuehn@aei.mpg.de> To: Marek Salwerowicz <marek_sal@wp.pl> Cc: freebsd-net@freebsd.org Subject: Re: NFS over LAGG / lacp poor performance Message-ID: <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> In-Reply-To: <535A1354.2040309@wp.pl> References: <535A1354.2040309@wp.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 25 Apr 2014 09:48:36 +0200 Marek Salwerowicz <marek_sal@wp.pl> wrote about NFS over LAGG / lacp poor performance: Hello Marek, MS> FreeBSD storage1 9.1-RELEASE-p10 FreeBSD 9.1-RELEASE-p10 #0: Sun Jan 12 MS> 20:11:23 UTC 2014 MS> root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Sorry for hijacking this (as I can probably not help you with your issue at hand), but I have a roughly similar setup here (9.1-REL nfs server on X9 mainboard with igb interfaces) and was wondering lately about some performance issues I have on a dedicated nfs link. While looking around on the system, I noticed that interrupts on the igb queues spread a bit strangely: --- root@storage:/root # vmstat -i interrupt total rate irq1: atkbd0 2743 0 irq18: ehci0 uhci5 4445137 2 irq21: uhci1 29 0 cpu0:timer 352597482 228 irq256: igb0:que 0 99396134 64 irq257: igb0:que 1 61496018 39 irq258: igb0:que 2 101687742 66 irq259: igb0:que 3 100824264 65 irq260: igb0:link 2 0 irq261: igb1:que 0 1666960 1 irq262: igb1:que 1 2325576555 1510 irq263: igb1:que 2 1563283 1 irq264: igb1:que 3 1897428 1 irq265: igb1:link 2 0 irq266: mps0 327734440 212 irq267: mps1 193113287 125 irq268: mps2 174367181 113 irq269: ahci0 59169140 38 cpu1:timer 416297615 270 cpu3:timer 327005767 212 cpu2:timer 325504623 211 Total 4874345832 3165 --- igb0 is only running standard/admin stuff, igb1 is running the mentioned dedicated nfs link. As you can see, igb1 gets interrupts on one queue only. I wonder if this is what it should be and if if has any impact on performace. I found several threads on the mailing lists about igb interfaces, queues and interrupts, but so far I could not really make out if this might be an issue for me. Could you have a look on your system and let me know how your interrupts are spread? cu Gerrit
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140425113711.e7c7d1c2.gerrit.kuehn>