Date: Mon, 3 Aug 2015 17:05:42 +0200 From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= <eri@freebsd.org> To: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= <olivier@cochard.me> Cc: Gleb Smirnoff <glebius@freebsd.org>, svn-src-head <svn-src-head@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, src-committers <src-committers@freebsd.org> Subject: Re: svn commit: r285051 - head/sys/netinet Message-ID: <CAPBZQG2fL9pwUr30jXt_EDp_YYvk6wjPOHxj7vpW0tJHcvyWGw@mail.gmail.com> In-Reply-To: <CA%2Bq%2BTcooswo3yNSgiicPG6SdcLAraxGMW8%2By9XTrcrCoFh=DVw@mail.gmail.com> References: <201507021810.t62IAgCc003272@repo.freebsd.org> <20150728124220.GW72729@FreeBSD.org> <CA%2Bq%2BTcooswo3yNSgiicPG6SdcLAraxGMW8%2By9XTrcrCoFh=DVw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello Olivier, its strange seeing so much contention on the arp tables on your PMC stats. Do you run ping(to prepopulate arp) or static arp to remove the noise from that interaction? Also do you run with flowtable active? On Mon, Aug 3, 2015 at 3:06 PM, Olivier Cochard-Labb=C3=A9 <olivier@cochard= .me> wrote: > On Tue, Jul 28, 2015 at 2:42 PM, Gleb Smirnoff <glebius@freebsd.org> > wrote: > >> >> Here is suggested patch. Ermal and Oliver, can you please test/benchmark >> it? >> > > =E2=80=8BHi, > > this patch reduce performanece :-( > > Here are the results regarding forwarding: > > x r285046.pps.forwarding (IPSEC compiled but not used) > + r285051.pps.forwarding (IPSEC compiled but not used) > * r285051-glebius-patched.pps.forwarding (IPSEC compiled but not used) > > +------------------------------------------------------------------------= --------+ > |xxx xx **** > * + +++| > ||_M_A___| > |_A_| |AM|| > > +------------------------------------------------------------------------= --------+ > N Min Max Median Avg Stdd= ev > x 5 397733 406951 399300 401613.8 4324.97= 55 > + 5 478095 482079 480869 480543.6 1666.02= 82 > Difference at 95.0% confidence > 78929.8 +/- 4779.72 > 19.6532% +/- 1.19013% > (Student's t, pooled s =3D 3277.27) > * 5 424720 430745 427014 427378.4 2351.74= 39 > Difference at 95.0% confidence > 25764.6 +/- 5076.98 > 6.41527% +/- 1.26415% > (Student's t, pooled s =3D 3481.1) > =E2=80=8B > > > PMC stats during forwarding bench: > [root@netgate]/data# pmcannotate pmc.forwarding.out > /data/debug/boot/kernel/kernel.symbols > CONVERSION STATISTICS: > #samples/total 33880 > Profile trace for function: __rw_rlock() [6.29%] > Profile trace for function: ip_forward() [4.68%] > Profile trace for function: ip_output() [4.64%] > Profile trace for function: binuptime() [4.05%] > Profile trace for function: igb_mq_start_locked() [3.79%] > Profile trace for function: igb_rxeof() [3.46%] > Profile trace for function: tsc_get_timecount_low_lfence() [3.25%] > Profile trace for function: ether_output() [3.03%] > Profile trace for function: rtalloc1_fib() [2.77%] > Profile trace for function: random_ivy_read() [2.64%] > Profile trace for function: _rw_runlock_cookie() [2.64%] > Profile trace for function: ether_nh_input() [2.63%] > Profile trace for function: ip_input() [2.55%] > Profile trace for function: key_allocsp_default() [2.39%] > Profile trace for function: igb_mq_start() [2.39%] > Profile trace for function: bzero() [2.08%] > Profile trace for function: uma_zalloc_arg() [1.95%] > Profile trace for function: memcpy() [1.84%] > Profile trace for function: _mtx_lock_spin_cookie() [1.83%] > Profile trace for function: bcopy() [1.76%] > Profile trace for function: random_harvest_queue() [1.63%] > Profile trace for function: __mtx_lock_sleep() [1.56%] > Profile trace for function: uma_zfree_arg() [1.47%] > Profile trace for function: arpresolve() [1.39%] > Profile trace for function: in_cksumdata() [1.25%] > Profile trace for function: bounce_bus_dmamap_load_buffer() [1.22%] > Profile trace for function: bcmp() [1.13%] > Profile trace for function: rtalloc_ign_fib() [1.11%] > Profile trace for function: rn_match() [1.03%] > Profile trace for function: netisr_dispatch_src() [1.03%] > Profile trace for function: critical_exit() [1.02%] > Profile trace for function: bus_dmamap_load_mbuf_sg() [0.87%] > Profile trace for function: spinlock_exit() [0.79%] > Profile trace for function: in_cksum_skip() [0.75%] > Profile trace for function: ip_ipsec_output() [0.75%] > Profile trace for function: acpi_cpu_c1() [0.74%] > Profile trace for function: in_broadcast() [0.74%] > Profile trace for function: spinlock_enter() [0.74%] > Profile trace for function: igb_refresh_mbufs() [0.71%] > Profile trace for function: in_lltable_lookup() [0.71%] > Profile trace for function: ip_fastforward() [0.68%] > Profile trace for function: m_adj() [0.65%] > Profile trace for function: ether_demux() [0.65%] > Profile trace for function: _key_freesp() [0.61%] > Profile trace for function: lockstat_nsecs() [0.60%] > Profile trace for function: m_freem() [0.58%] > Profile trace for function: critical_enter() [0.56%] > Profile trace for function: m_copydata() [0.55%] > Profile trace for function: mb_free_ext() [0.54%] > Profile trace for function: pmap_kextract() [0.50%] > > =E2=80=8B > > =E2=80=8BAnd about fastforwarding: > =E2=80=8B > x 285046.pps.fastforwarding (IPSEC compiled but not used) > + 285051.pps.fastforwarding (IPSEC compiled but not used) > * r285051-glebius-patched.pps.fastforwarding (IPSEC compiled but not used= ) > > +------------------------------------------------------------------------= --------+ > |* > + | > |* * * * ++ +x > x+ xx x| > ||____AM__| > |_MA__|__AM_| | > > +------------------------------------------------------------------------= --------+ > N Min Max Median Avg Stdd= ev > x 5 742683 754709 750940 749015.6 4944.90= 59 > + 5 736459 747593 738197 740112.4 4511.16= 11 > Difference at 95.0% confidence > -8903.2 +/- 6902.82 > -1.18865% +/- 0.921585% > (Student's t, pooled s =3D 4733) > * 5 638356 652855 646668 645263 6599.86= 04 > Difference at 95.0% confidence > -103753 +/- 8504.75 > -13.8519% +/- 1.13546% > (Student's t, pooled s =3D 5831.39) > > And pmcstat during fastforwarding: > > [root@netgate]/data# pmcannotate pmc.fastforwading.out > /data/debug/boot/kernel/kernel.symbols > CONVERSION STATISTICS: > #samples/total 37656 > Profile trace for function: __rw_rlock() [9.12%] > Profile trace for function: ip_fastforward() [5.80%] > Profile trace for function: binuptime() [5.73%] > Profile trace for function: rtalloc1_fib() [4.92%] > Profile trace for function: tsc_get_timecount_low_lfence() [4.71%] > Profile trace for function: igb_rxeof() [4.51%] > Profile trace for function: _rw_runlock_cookie() [4.14%] > Profile trace for function: igb_mq_start_locked() [4.12%] > Profile trace for function: ether_output() [3.36%] > Profile trace for function: ether_nh_input() [3.11%] > Profile trace for function: igb_mq_start() [3.02%] > Profile trace for function: random_ivy_read() [2.88%] > Profile trace for function: bzero() [2.70%] > Profile trace for function: memcpy() [2.54%] > Profile trace for function: random_harvest_queue() [2.32%] > Profile trace for function: __mtx_lock_sleep() [1.91%] > Profile trace for function: bcmp() [1.78%] > Profile trace for function: _mtx_lock_spin_cookie() [1.73%] > Profile trace for function: igb_txeof() [1.70%] > Profile trace for function: rn_match() [1.69%] > Profile trace for function: bounce_bus_dmamap_load_buffer() [1.52%] > Profile trace for function: rtalloc_ign_fib() [1.30%] > Profile trace for function: mb_free_ext() [1.29%] > Profile trace for function: arpresolve() [1.28%] > Profile trace for function: bcopy() [1.22%] > Profile trace for function: spinlock_exit() [1.06%] > Profile trace for function: mb_ctor_pack() [1.04%] > Profile trace for function: uma_zalloc_arg() [1.02%] > Profile trace for function: spinlock_enter() [1.00%] > Profile trace for function: bus_dmamap_load_mbuf_sg() [0.93%] > Profile trace for function: critical_exit() [0.90%] > Profile trace for function: igb_refresh_mbufs() [0.89%] > Profile trace for function: acpi_cpu_c1() [0.89%] > Profile trace for function: netisr_dispatch_src() [0.81%] > Profile trace for function: uma_zfree_arg() [0.80%] > Profile trace for function: lockstat_nsecs() [0.79%] > Profile trace for function: ether_demux() [0.76%] > Profile trace for function: in_lltable_lookup() [0.75%] > Profile trace for function: ip_findroute() [0.66%] > Profile trace for function: in_localip() [0.65%] > Profile trace for function: _mtx_trylock_flags_() [0.57%] > Profile trace for function: _bus_dmamap_load_buffer() [0.54%] > > =E2=80=8BRegards, > > Olivier=E2=80=8B > --=20 Ermal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPBZQG2fL9pwUr30jXt_EDp_YYvk6wjPOHxj7vpW0tJHcvyWGw>