From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 01:12:44 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C849106566B; Tue, 24 Apr 2012 01:12:44 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E39FF8FC14; Tue, 24 Apr 2012 01:12:43 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q3O1ChOM023634; Tue, 24 Apr 2012 01:12:43 GMT (envelope-from np@svn.freebsd.org) Received: (from np@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q3O1ChWm023631; Tue, 24 Apr 2012 01:12:43 GMT (envelope-from np@svn.freebsd.org) Message-Id: <201204240112.q3O1ChWm023631@svn.freebsd.org> From: Navdeep Parhar Date: Tue, 24 Apr 2012 01:12:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r234627 - in user/np/toe_iwarp/sys/ofed: drivers/infiniband/core include/linux X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 01:12:44 -0000 Author: np Date: Tue Apr 24 01:12:43 2012 New Revision: 234627 URL: http://svn.freebsd.org/changeset/base/234627 Log: sock_getname: check whether connected only if the peer's name has been requested. Modified: user/np/toe_iwarp/sys/ofed/drivers/infiniband/core/cma.c user/np/toe_iwarp/sys/ofed/include/linux/net.h Modified: user/np/toe_iwarp/sys/ofed/drivers/infiniband/core/cma.c ============================================================================== --- user/np/toe_iwarp/sys/ofed/drivers/infiniband/core/cma.c Tue Apr 24 00:13:59 2012 (r234626) +++ user/np/toe_iwarp/sys/ofed/drivers/infiniband/core/cma.c Tue Apr 24 01:12:43 2012 (r234627) @@ -2257,12 +2257,7 @@ static int cma_get_tcp_port(struct rdma_ sock_release(sock); return ret; } -/* - * sock_getname does not seem to work here and it seems more similar to getpeername. - * Instead something like kern_getsockname should be used. Till that change is done - * lets disable the below code. - */ -#if 0 + size = ip_addr_size((struct sockaddr *) &id_priv->id.route.addr.src_addr); ret = sock_getname(sock, (struct sockaddr *) &id_priv->id.route.addr.src_addr, @@ -2271,7 +2266,7 @@ static int cma_get_tcp_port(struct rdma_ sock_release(sock); return ret; } -#endif + id_priv->sock = sock; return 0; } Modified: user/np/toe_iwarp/sys/ofed/include/linux/net.h ============================================================================== --- user/np/toe_iwarp/sys/ofed/include/linux/net.h Tue Apr 24 00:13:59 2012 (r234626) +++ user/np/toe_iwarp/sys/ofed/include/linux/net.h Tue Apr 24 01:12:43 2012 (r234627) @@ -48,12 +48,12 @@ sock_getname(struct socket *so, struct s int error; nam = NULL; - if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) - return (-ENOTCONN); + if (peer) { + if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) + return (-ENOTCONN); - if (peer) error = (*so->so_proto->pr_usrreqs->pru_peeraddr)(so, nam); - else + } else error = (*so->so_proto->pr_usrreqs->pru_sockaddr)(so, nam); if (error) return (-error); From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 11:46:16 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4D43F106566B; Tue, 24 Apr 2012 11:46:16 +0000 (UTC) (envelope-from andre@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 1FA6F8FC0A; Tue, 24 Apr 2012 11:46:16 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q3OBkFDg047130; Tue, 24 Apr 2012 11:46:15 GMT (envelope-from andre@svn.freebsd.org) Received: (from andre@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q3OBkFoA047129; Tue, 24 Apr 2012 11:46:15 GMT (envelope-from andre@svn.freebsd.org) Message-Id: <201204241146.q3OBkFoA047129@svn.freebsd.org> From: Andre Oppermann Date: Tue, 24 Apr 2012 11:46:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r234647 - user/andre/routelocking X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 11:46:16 -0000 Author: andre Date: Tue Apr 24 11:46:15 2012 New Revision: 234647 URL: http://svn.freebsd.org/changeset/base/234647 Log: Add experimental branch for testing of different routing table locking strategies. Not intended to merged back to HEAD. Added: - copied from r234646, head/sys/ Directory Properties: user/andre/routelocking/ (props changed) From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 12:15:51 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A4D7106566C; Tue, 24 Apr 2012 12:15:51 +0000 (UTC) (envelope-from andre@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 34D278FC08; Tue, 24 Apr 2012 12:15:51 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q3OCFprA048139; Tue, 24 Apr 2012 12:15:51 GMT (envelope-from andre@svn.freebsd.org) Received: (from andre@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q3OCFoYf048135; Tue, 24 Apr 2012 12:15:50 GMT (envelope-from andre@svn.freebsd.org) Message-Id: <201204241215.q3OCFoYf048135@svn.freebsd.org> From: Andre Oppermann Date: Tue, 24 Apr 2012 12:15:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r234648 - in user/andre/routelocking: kern sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 12:15:51 -0000 Author: andre Date: Tue Apr 24 12:15:50 2012 New Revision: 234648 URL: http://svn.freebsd.org/changeset/base/234648 Log: Add INVARIANT and WITNESS support to rm_lock locks and optimize the synchronization path by replacing a LIST of active readers with a TAILQ. Obtained from: Isilon Submitted by: mlaier Modified: user/andre/routelocking/kern/kern_rmlock.c user/andre/routelocking/sys/_rmlock.h user/andre/routelocking/sys/rmlock.h Modified: user/andre/routelocking/kern/kern_rmlock.c ============================================================================== --- user/andre/routelocking/kern/kern_rmlock.c Tue Apr 24 11:46:15 2012 (r234647) +++ user/andre/routelocking/kern/kern_rmlock.c Tue Apr 24 12:15:50 2012 (r234648) @@ -102,15 +102,25 @@ assert_rm(const struct lock_object *lock static void lock_rm(struct lock_object *lock, int how) { + struct rmlock *rm; - panic("lock_rm called"); + rm = (struct rmlock *)lock; + if (how) + rm_wlock(rm); + else + panic("lock_rm called in shared mode"); } static int unlock_rm(struct lock_object *lock) { + struct rmlock *rm; - panic("unlock_rm called"); + rm = (struct rmlock *)lock; + if (!rm_wowned(rm)) + panic("unlock_rm called without exclusive lock held"); + rm_wunlock(rm); + return (1); } #ifdef KDTRACE_HOOKS @@ -167,6 +177,7 @@ rm_tracker_remove(struct pcpu *pc, struc static void rm_cleanIPI(void *arg) { + TAILQ_HEAD(,rm_priotracker) tmp_list = TAILQ_HEAD_INITIALIZER(tmp_list); struct pcpu *pc; struct rmlock *rm = arg; struct rm_priotracker *tracker; @@ -178,12 +189,12 @@ rm_cleanIPI(void *arg) tracker = (struct rm_priotracker *)queue; if (tracker->rmp_rmlock == rm && tracker->rmp_flags == 0) { tracker->rmp_flags = RMPF_ONQUEUE; - mtx_lock_spin(&rm_spinlock); - LIST_INSERT_HEAD(&rm->rm_activeReaders, tracker, - rmp_qentry); - mtx_unlock_spin(&rm_spinlock); + TAILQ_INSERT_HEAD(&tmp_list, tracker, rmp_qentry); } } + mtx_lock_spin(&rm_spinlock); + TAILQ_CONCAT(&rm->rm_activeReaders, &tmp_list, rmp_qentry); + mtx_unlock_spin(&rm_spinlock); } CTASSERT((RM_SLEEPABLE & LO_CLASSFLAGS) == RM_SLEEPABLE); @@ -199,7 +210,7 @@ rm_init_flags(struct rmlock *rm, const c if (opts & RM_RECURSE) liflags |= LO_RECURSABLE; rm->rm_writecpus = all_cpus; - LIST_INIT(&rm->rm_activeReaders); + TAILQ_INIT(&rm->rm_activeReaders); if (opts & RM_SLEEPABLE) { liflags |= RM_SLEEPABLE; sx_init_flags(&rm->rm_lock_sx, "rmlock_sx", SX_RECURSE); @@ -227,7 +238,7 @@ rm_destroy(struct rmlock *rm) } int -rm_wowned(const struct rmlock *rm) +rm_wowned(struct rmlock *rm) { if (rm->lock_object.lo_flags & RM_SLEEPABLE) @@ -294,7 +305,7 @@ _rm_rlock_hard(struct rmlock *rm, struct if ((atracker->rmp_rmlock == rm) && (atracker->rmp_thread == tracker->rmp_thread)) { mtx_lock_spin(&rm_spinlock); - LIST_INSERT_HEAD(&rm->rm_activeReaders, + TAILQ_INSERT_HEAD(&rm->rm_activeReaders, tracker, rmp_qentry); tracker->rmp_flags = RMPF_ONQUEUE; mtx_unlock_spin(&rm_spinlock); @@ -390,7 +401,8 @@ _rm_unlock_hard(struct thread *td,struct return; mtx_lock_spin(&rm_spinlock); - LIST_REMOVE(tracker, rmp_qentry); + TAILQ_REMOVE(&tracker->rmp_rmlock->rm_activeReaders, tracker, + rmp_qentry); if (tracker->rmp_flags & RMPF_SIGNAL) { struct rmlock *rm; @@ -468,7 +480,7 @@ _rm_wlock(struct rmlock *rm) #endif mtx_lock_spin(&rm_spinlock); - while ((prio = LIST_FIRST(&rm->rm_activeReaders)) != NULL) { + while ((prio = TAILQ_FIRST(&rm->rm_activeReaders)) != NULL) { ts = turnstile_trywait(&rm->lock_object); prio->rmp_flags = RMPF_ONQUEUE | RMPF_SIGNAL; mtx_unlock_spin(&rm_spinlock); @@ -609,3 +621,92 @@ _rm_runlock_debug(struct rmlock *rm, str } #endif + +#ifdef INVARIANT_SUPPORT +void +_rm_assert(struct rmlock *rm, int what, const char *file, int line) +{ +#ifndef WITNESS + struct pcpu *pc; + struct rm_queue *queue; + struct rm_priotracker *atracker; + int cnt; +#endif + + if (panicstr != NULL) + return; + switch (what) { + case RM_LOCKED: + case RM_LOCKED | RM_RECURSED: + case RM_LOCKED | RM_NOTRECURSED: + case RM_RLOCKED | RM_RECURSED: + case RM_RLOCKED | RM_NOTRECURSED: + case RM_RLOCKED: +#ifdef WITNESS + witness_assert(&rm->lock_object, what, file, line); +#else + if ((what == RM_RLOCKED) && rm_wowned(rm)) + panic("Lock %s writelocked @ %s:%d\n", + rm->lock_object.lo_name, file, line); + + critical_enter(); + pc = pcpu_find(curcpu); + cnt = 0; + for (queue = pc->pc_rm_queue.rmq_next; + queue != &pc->pc_rm_queue; queue = queue->rmq_next) { + atracker = (struct rm_priotracker *)queue; + if ((atracker->rmp_rmlock == rm) && + (atracker->rmp_thread == curthread)) + cnt++; + } + critical_exit(); + + if ((cnt == 0) && !rm_wowned(rm)) + panic("Lock %s not %slocked @ %s:%d\n", + rm->lock_object.lo_name, (what == RM_RLOCKED) ? + "read " : "", file, line); + if (cnt > 2) { + if (what & RA_NOTRECURSED) + panic("Lock %s recursed @ %s:%d\n", + rm->lock_object.lo_name, file, line); + } else if (what & RA_RECURSED) + panic("Lock %s not recursed @ %s:%d\n", + rm->lock_object.lo_name, file, line); +#endif + break; + case RM_WLOCKED: + if (!rm_wowned(rm)) + panic("Lock %s not writelocked @ %s:%d\n", + rm->lock_object.lo_name, file, line); + break; + case RM_UNLOCKED: +#ifdef WITNESS + witness_assert(&rm->lock_object, what, file, line); +#else + if (rm_wowned(rm)) + panic("Lock %s writelocked @ %s:%d\n", + rm->lock_object.lo_name, file, line); + + critical_enter(); + pc = pcpu_find(curcpu); + cnt = 0; + for (queue = pc->pc_rm_queue.rmq_next; + queue != &pc->pc_rm_queue; queue = queue->rmq_next) { + atracker = (struct rm_priotracker *)queue; + if ((atracker->rmp_rmlock == rm) && + (atracker->rmp_thread == curthread)) + cnt++; + } + critical_exit(); + + if (cnt != 0) + panic("Lock %s readlocked @ %s:%d\n", + rm->lock_object.lo_name, file, line); +#endif + break; + default: + panic("Unknown rm lock assertion: %d @ %s:%d", what, file, + line); + } +} +#endif /* INVARIANT_SUPPORT */ Modified: user/andre/routelocking/sys/_rmlock.h ============================================================================== --- user/andre/routelocking/sys/_rmlock.h Tue Apr 24 11:46:15 2012 (r234647) +++ user/andre/routelocking/sys/_rmlock.h Tue Apr 24 12:15:50 2012 (r234648) @@ -35,18 +35,18 @@ /* * XXXUPS remove as soon as we have per cpu variable * linker sets and can define rm_queue in _rm_lock.h -*/ + */ #include /* * Mostly reader/occasional writer lock. */ -LIST_HEAD(rmpriolist,rm_priotracker); +TAILQ_HEAD(rmpriolist, rm_priotracker); struct rmlock { struct lock_object lock_object; volatile cpuset_t rm_writecpus; - LIST_HEAD(,rm_priotracker) rm_activeReaders; + TAILQ_HEAD(, rm_priotracker) rm_activeReaders; union { struct mtx _rm_lock_mtx; struct sx _rm_lock_sx; @@ -60,7 +60,7 @@ struct rm_priotracker { struct rmlock *rmp_rmlock; struct thread *rmp_thread; int rmp_flags; - LIST_ENTRY(rm_priotracker) rmp_qentry; + TAILQ_ENTRY(rm_priotracker) rmp_qentry; }; #endif /* !_SYS__RMLOCK_H_ */ Modified: user/andre/routelocking/sys/rmlock.h ============================================================================== --- user/andre/routelocking/sys/rmlock.h Tue Apr 24 11:46:15 2012 (r234647) +++ user/andre/routelocking/sys/rmlock.h Tue Apr 24 12:15:50 2012 (r234648) @@ -49,7 +49,7 @@ void rm_init(struct rmlock *rm, const char *name); void rm_init_flags(struct rmlock *rm, const char *name, int opts); void rm_destroy(struct rmlock *rm); -int rm_wowned(const struct rmlock *rm); +int rm_wowned(struct rmlock *rm); void rm_sysinit(void *arg); void rm_sysinit_flags(void *arg); @@ -65,6 +65,9 @@ void _rm_wunlock(struct rmlock *rm); int _rm_rlock(struct rmlock *rm, struct rm_priotracker *tracker, int trylock); void _rm_runlock(struct rmlock *rm, struct rm_priotracker *tracker); +#if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) +void _rm_assert(struct rmlock *rm, int what, const char *file, int line); +#endif /* * Public interface for lock operations. @@ -89,6 +92,8 @@ void _rm_runlock(struct rmlock *rm, str #define rm_try_rlock(rm,tracker) _rm_rlock((rm),(tracker), 1) #define rm_runlock(rm,tracker) _rm_runlock((rm), (tracker)) #endif +#define rm_sleep(chan, rm, pri, wmesg, timo) \ + _sleep((chan), &(rm)->lock_object, (pri), (wmesg), (timo)) struct rm_args { struct rmlock *ra_rm; @@ -123,5 +128,20 @@ struct rm_args_flags { SYSUNINIT(name##_rm_sysuninit, SI_SUB_LOCK, SI_ORDER_MIDDLE, \ rm_destroy, (rm)) +#if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) +#define RM_LOCKED LA_LOCKED +#define RM_RLOCKED LA_SLOCKED +#define RM_WLOCKED LA_XLOCKED +#define RM_UNLOCKED LA_UNLOCKED +#define RM_RECURSED LA_RECURSED +#define RM_NOTRECURSED LA_NOTRECURSED +#endif + +#ifdef INVARIANTS +#define rm_assert(rm, what) _rm_assert((rm), (what), LOCK_FILE, LOCK_LINE) +#else +#define rm_assert(rm, what) +#endif + #endif /* _KERNEL */ #endif /* !_SYS_RMLOCK_H_ */ From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 12:43:30 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D0B5F106566B; Tue, 24 Apr 2012 12:43:30 +0000 (UTC) (envelope-from andre@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BA4318FC17; Tue, 24 Apr 2012 12:43:30 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q3OChUl9048978; Tue, 24 Apr 2012 12:43:30 GMT (envelope-from andre@svn.freebsd.org) Received: (from andre@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q3OChUqE048963; Tue, 24 Apr 2012 12:43:30 GMT (envelope-from andre@svn.freebsd.org) Message-Id: <201204241243.q3OChUqE048963@svn.freebsd.org> From: Andre Oppermann Date: Tue, 24 Apr 2012 12:43:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r234649 - in user/andre/routelocking: contrib/ipfilter/netinet contrib/pf/net kern net netinet netinet6 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 12:43:30 -0000 Author: andre Date: Tue Apr 24 12:43:29 2012 New Revision: 234649 URL: http://svn.freebsd.org/changeset/base/234649 Log: hange the radix head lock to an rmlock (read mostly lock). There is some header pollution going on because rmlock's are not entirely abstracted and need per-CPU structures. A comment in _rmlock.h says this can be hidden if there were per-cpu linker magic/support. I don't know if we have that already. Modified: user/andre/routelocking/contrib/ipfilter/netinet/ip_pool.c user/andre/routelocking/contrib/pf/net/pf_table.c user/andre/routelocking/kern/subr_witness.c user/andre/routelocking/kern/vfs_export.c user/andre/routelocking/net/if.c user/andre/routelocking/net/radix.c user/andre/routelocking/net/radix.h user/andre/routelocking/net/radix_mpath.c user/andre/routelocking/net/route.c user/andre/routelocking/net/rtsock.c user/andre/routelocking/netinet/in_rmx.c user/andre/routelocking/netinet6/in6_ifattach.c user/andre/routelocking/netinet6/in6_rmx.c user/andre/routelocking/netinet6/nd6_rtr.c Modified: user/andre/routelocking/contrib/ipfilter/netinet/ip_pool.c ============================================================================== --- user/andre/routelocking/contrib/ipfilter/netinet/ip_pool.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/contrib/ipfilter/netinet/ip_pool.c Tue Apr 24 12:43:29 2012 (r234649) @@ -51,6 +51,8 @@ struct file; #endif #if defined(__FreeBSD_version) && (__FreeBSD_version >= 300000) # include +# include +# include #endif #if defined(SOLARIS2) && !defined(_KERNEL) Modified: user/andre/routelocking/contrib/pf/net/pf_table.c ============================================================================== --- user/andre/routelocking/contrib/pf/net/pf_table.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/contrib/pf/net/pf_table.c Tue Apr 24 12:43:29 2012 (r234649) @@ -45,6 +45,8 @@ __FBSDID("$FreeBSD$"); #include #ifdef __FreeBSD__ #include +#include +#include #else #include #endif Modified: user/andre/routelocking/kern/subr_witness.c ============================================================================== --- user/andre/routelocking/kern/subr_witness.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/kern/subr_witness.c Tue Apr 24 12:43:29 2012 (r234649) @@ -509,7 +509,7 @@ static struct witness_order_list_entry o * Routing */ { "so_rcv", &lock_class_mtx_sleep }, - { "radix node head", &lock_class_rw }, + { "radix node head", &lock_class_rm }, { "rtentry", &lock_class_mtx_sleep }, { "ifaddr", &lock_class_mtx_sleep }, { NULL, NULL }, Modified: user/andre/routelocking/kern/vfs_export.c ============================================================================== --- user/andre/routelocking/kern/vfs_export.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/kern/vfs_export.c Tue Apr 24 12:43:29 2012 (r234649) @@ -43,6 +43,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -427,6 +428,7 @@ vfs_export_lookup(struct mount *mp, stru register struct netcred *np; register struct radix_node_head *rnh; struct sockaddr *saddr; + struct rm_priotracker tracker; nep = mp->mnt_export; if (nep == NULL) @@ -440,10 +442,10 @@ vfs_export_lookup(struct mount *mp, stru saddr = nam; rnh = nep->ne_rtable[saddr->sa_family]; if (rnh != NULL) { - RADIX_NODE_HEAD_RLOCK(rnh); + RADIX_NODE_HEAD_RLOCK(rnh, &tracker); np = (struct netcred *) (*rnh->rnh_matchaddr)(saddr, rnh); - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); if (np && np->netc_rnodes->rn_flags & RNF_ROOT) np = NULL; } Modified: user/andre/routelocking/net/if.c ============================================================================== --- user/andre/routelocking/net/if.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/if.c Tue Apr 24 12:43:29 2012 (r234649) @@ -49,6 +49,7 @@ #include #include #include +#include #include #include #include Modified: user/andre/routelocking/net/radix.c ============================================================================== --- user/andre/routelocking/net/radix.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/radix.c Tue Apr 24 12:43:29 2012 (r234649) @@ -37,7 +37,7 @@ #ifdef _KERNEL #include #include -#include +#include #include #include #include Modified: user/andre/routelocking/net/radix.h ============================================================================== --- user/andre/routelocking/net/radix.h Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/radix.h Tue Apr 24 12:43:29 2012 (r234649) @@ -36,7 +36,7 @@ #ifdef _KERNEL #include #include -#include +#include #endif #ifdef MALLOC_DECLARE @@ -134,7 +134,7 @@ struct radix_node_head { (struct radix_node *rn, struct radix_node_head *head); struct radix_node rnh_nodes[3]; /* empty tree for common case */ #ifdef _KERNEL - struct rwlock rnh_lock; /* locks entire radix tree */ + struct rmlock rnh_lock; /* locks entire radix tree */ #endif }; @@ -148,17 +148,16 @@ struct radix_node_head { #define Free(p) free((caddr_t)p, M_RTABLE); #define RADIX_NODE_HEAD_LOCK_INIT(rnh) \ - rw_init_flags(&(rnh)->rnh_lock, "radix node head", 0) -#define RADIX_NODE_HEAD_LOCK(rnh) rw_wlock(&(rnh)->rnh_lock) -#define RADIX_NODE_HEAD_UNLOCK(rnh) rw_wunlock(&(rnh)->rnh_lock) -#define RADIX_NODE_HEAD_RLOCK(rnh) rw_rlock(&(rnh)->rnh_lock) -#define RADIX_NODE_HEAD_RUNLOCK(rnh) rw_runlock(&(rnh)->rnh_lock) -#define RADIX_NODE_HEAD_LOCK_TRY_UPGRADE(rnh) rw_try_upgrade(&(rnh)->rnh_lock) + rm_init_flags(&(rnh)->rnh_lock, "radix node head", 0) +#define RADIX_NODE_HEAD_DESTROY(rnh) rm_destroy(&(rnh)->rnh_lock) +#define RADIX_NODE_HEAD_LOCK(rnh) rm_wlock(&(rnh)->rnh_lock) +#define RADIX_NODE_HEAD_UNLOCK(rnh) rm_wunlock(&(rnh)->rnh_lock) +#define RADIX_NODE_HEAD_RLOCK(rnh, tracker) rm_rlock(&(rnh)->rnh_lock, (tracker)) +#define RADIX_NODE_HEAD_RUNLOCK(rnh, tracker) rm_runlock(&(rnh)->rnh_lock, (tracker)) -#define RADIX_NODE_HEAD_DESTROY(rnh) rw_destroy(&(rnh)->rnh_lock) -#define RADIX_NODE_HEAD_LOCK_ASSERT(rnh) rw_assert(&(rnh)->rnh_lock, RA_LOCKED) -#define RADIX_NODE_HEAD_WLOCK_ASSERT(rnh) rw_assert(&(rnh)->rnh_lock, RA_WLOCKED) +#define RADIX_NODE_HEAD_LOCK_ASSERT(rnh) rm_assert(&(rnh)->rnh_lock, RM_WLOCKED) +#define RADIX_NODE_HEAD_RLOCK_ASSERT(rnh) rm_assert(&(rnh)->rnh_lock, RM_RLOCKED) #endif /* _KERNEL */ void rn_init(int); Modified: user/andre/routelocking/net/radix_mpath.c ============================================================================== --- user/andre/routelocking/net/radix_mpath.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/radix_mpath.c Tue Apr 24 12:43:29 2012 (r234649) @@ -45,6 +45,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include #include Modified: user/andre/routelocking/net/route.c ============================================================================== --- user/andre/routelocking/net/route.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/route.c Tue Apr 24 12:43:29 2012 (r234649) @@ -52,6 +52,8 @@ #include #include #include +#include +#include #include #include @@ -339,7 +341,7 @@ rtalloc_ign_fib(struct route *ro, u_long /* * Look up the route that matches the address given - * Or, at least try.. Create a cloned route if needed. + * Or, at least try. * * The returned route, if any, is locked. */ @@ -351,13 +353,13 @@ rtalloc1(struct sockaddr *dst, int repor } struct rtentry * -rtalloc1_fib(struct sockaddr *dst, int report, u_long ignflags, - u_int fibnum) +rtalloc1_fib(struct sockaddr *dst, int report, u_long ignflags, u_int fibnum) { struct radix_node_head *rnh; struct radix_node *rn; struct rtentry *newrt; struct rt_addrinfo info; + struct rm_priotracker tracker; int err = 0, msgtype = RTM_MISS; int needlock; @@ -381,22 +383,23 @@ rtalloc1_fib(struct sockaddr *dst, int r */ needlock = !(ignflags & RTF_RNH_LOCKED); if (needlock) - RADIX_NODE_HEAD_RLOCK(rnh); -#ifdef INVARIANTS + RADIX_NODE_HEAD_RLOCK(rnh, &tracker); +#ifdef INVARIANTS else - RADIX_NODE_HEAD_LOCK_ASSERT(rnh); + RADIX_NODE_HEAD_RLOCK_ASSERT(rnh); #endif + rn = rnh->rnh_matchaddr(dst, rnh); if (rn && ((rn->rn_flags & RNF_ROOT) == 0)) { newrt = RNTORT(rn); RT_LOCK(newrt); RT_ADDREF(newrt); if (needlock) - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); goto done; } else if (needlock) - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); /* * Either we hit the root or couldn't find any match, @@ -1085,8 +1088,11 @@ rtrequest1_fib(int req, struct rt_addrin flags &= ~RTF_RNH_LOCKED; if (needlock) RADIX_NODE_HEAD_LOCK(rnh); +#ifdef INVARIANTS else RADIX_NODE_HEAD_LOCK_ASSERT(rnh); +#endif + /* * If we are adding a host route then we don't want to put * a netmask in the tree, nor do we want to clone it. Modified: user/andre/routelocking/net/rtsock.c ============================================================================== --- user/andre/routelocking/net/rtsock.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/net/rtsock.c Tue Apr 24 12:43:29 2012 (r234649) @@ -51,6 +51,7 @@ #include #include #include +#include #include #include @@ -576,6 +577,7 @@ route_output(struct mbuf *m, struct sock int len, error = 0; struct ifnet *ifp = NULL; union sockaddr_union saun; + struct rm_priotracker tracker; sa_family_t saf = AF_UNSPEC; #define senderr(e) { error = e; goto flush;} @@ -708,11 +710,11 @@ route_output(struct mbuf *m, struct sock info.rti_info[RTAX_DST]->sa_family); if (rnh == NULL) senderr(EAFNOSUPPORT); - RADIX_NODE_HEAD_RLOCK(rnh); + RADIX_NODE_HEAD_RLOCK(rnh, &tracker); rt = (struct rtentry *) rnh->rnh_lookup(info.rti_info[RTAX_DST], info.rti_info[RTAX_NETMASK], rnh); if (rt == NULL) { /* XXX looks bogus */ - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); senderr(ESRCH); } #ifdef RADIX_MPATH @@ -728,7 +730,7 @@ route_output(struct mbuf *m, struct sock (rtm->rtm_type != RTM_GET || info.rti_info[RTAX_GATEWAY])) { rt = rt_mpath_matchgate(rt, info.rti_info[RTAX_GATEWAY]); if (!rt) { - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); senderr(ESRCH); } } @@ -760,13 +762,13 @@ route_output(struct mbuf *m, struct sock */ rt = (struct rtentry *)rnh->rnh_matchaddr(&laddr, rnh); if (rt == NULL) { - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); senderr(ESRCH); } } RT_LOCK(rt); RT_ADDREF(rt); - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); /* * Fix for PR: 82974 @@ -1838,6 +1840,7 @@ sysctl_rtsock(SYSCTL_HANDLER_ARGS) int i, lim, error = EINVAL; u_char af; struct walkarg w; + struct rm_priotracker tracker; name ++; namelen--; @@ -1884,10 +1887,10 @@ sysctl_rtsock(SYSCTL_HANDLER_ARGS) for (error = 0; error == 0 && i <= lim; i++) { rnh = rt_tables_get_rnh(req->td->td_proc->p_fibnum, i); if (rnh != NULL) { - RADIX_NODE_HEAD_RLOCK(rnh); + RADIX_NODE_HEAD_RLOCK(rnh, &tracker); error = rnh->rnh_walktree(rnh, sysctl_dumpentry, &w); - RADIX_NODE_HEAD_RUNLOCK(rnh); + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); } else if (af != 0) error = EAFNOSUPPORT; } Modified: user/andre/routelocking/netinet/in_rmx.c ============================================================================== --- user/andre/routelocking/netinet/in_rmx.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/netinet/in_rmx.c Tue Apr 24 12:43:29 2012 (r234649) @@ -51,6 +51,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include @@ -77,7 +79,7 @@ in_addroute(void *v_arg, void *n_arg, st struct rtentry *rt = (struct rtentry *)treenodes; struct sockaddr_in *sin = (struct sockaddr_in *)rt_key(rt); - RADIX_NODE_HEAD_WLOCK_ASSERT(head); + RADIX_NODE_HEAD_LOCK_ASSERT(head); /* * A little bit of help for both IP output and input: * For host routes, we make sure that RTF_BROADCAST @@ -205,7 +207,7 @@ in_rtqkill(struct radix_node *rn, void * struct rtentry *rt = (struct rtentry *)rn; int err; - RADIX_NODE_HEAD_WLOCK_ASSERT(ap->rnh); + RADIX_NODE_HEAD_LOCK_ASSERT(ap->rnh); if (rt->rt_flags & RTPRF_OURS) { ap->found++; Modified: user/andre/routelocking/netinet6/in6_ifattach.c ============================================================================== --- user/andre/routelocking/netinet6/in6_ifattach.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/netinet6/in6_ifattach.c Tue Apr 24 12:43:29 2012 (r234649) @@ -42,6 +42,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include Modified: user/andre/routelocking/netinet6/in6_rmx.c ============================================================================== --- user/andre/routelocking/netinet6/in6_rmx.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/netinet6/in6_rmx.c Tue Apr 24 12:43:29 2012 (r234649) @@ -87,6 +87,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -124,7 +125,7 @@ in6_addroute(void *v_arg, void *n_arg, s struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)rt_key(rt); struct radix_node *ret; - RADIX_NODE_HEAD_WLOCK_ASSERT(head); + RADIX_NODE_HEAD_LOCK_ASSERT(head); if (IN6_IS_ADDR_MULTICAST(&sin6->sin6_addr)) rt->rt_flags |= RTF_MULTICAST; @@ -247,7 +248,7 @@ in6_rtqkill(struct radix_node *rn, void struct rtentry *rt = (struct rtentry *)rn; int err; - RADIX_NODE_HEAD_WLOCK_ASSERT(ap->rnh); + RADIX_NODE_HEAD_LOCK_ASSERT(ap->rnh); if (rt->rt_flags & RTPRF_OURS) { ap->found++; Modified: user/andre/routelocking/netinet6/nd6_rtr.c ============================================================================== --- user/andre/routelocking/netinet6/nd6_rtr.c Tue Apr 24 12:15:50 2012 (r234648) +++ user/andre/routelocking/netinet6/nd6_rtr.c Tue Apr 24 12:43:29 2012 (r234649) @@ -48,6 +48,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 12:54:05 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 90A5B106566B; Tue, 24 Apr 2012 12:54:05 +0000 (UTC) (envelope-from andre@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 7A4A38FC08; Tue, 24 Apr 2012 12:54:05 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q3OCs5LP049321; Tue, 24 Apr 2012 12:54:05 GMT (envelope-from andre@svn.freebsd.org) Received: (from andre@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q3OCs5wo049315; Tue, 24 Apr 2012 12:54:05 GMT (envelope-from andre@svn.freebsd.org) Message-Id: <201204241254.q3OCs5wo049315@svn.freebsd.org> From: Andre Oppermann Date: Tue, 24 Apr 2012 12:54:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r234650 - in user/andre/routelocking: net netinet X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 12:54:05 -0000 Author: andre Date: Tue Apr 24 12:54:04 2012 New Revision: 234650 URL: http://svn.freebsd.org/changeset/base/234650 Log: Add a function rtlookup() that copies out the relevant information from an rtentry instead of returning the rtentry. This avoids the need to lock the rtentry and to increase the refcount on it. Convert ip_output() to use rtlookup() in a simplistic way. Certain seldom used functionality may not work anymore and the flowtable isn't available at the moment. Convert ip_fastfwd() to use rtlookup(). This code is meant to be used for profiling and to be experimented with further to determine which locking strategy returns the best results. Modified: user/andre/routelocking/net/route.c user/andre/routelocking/net/route.h user/andre/routelocking/netinet/ip_fastfwd.c user/andre/routelocking/netinet/ip_output.c user/andre/routelocking/netinet/tcp_output.c Modified: user/andre/routelocking/net/route.c ============================================================================== --- user/andre/routelocking/net/route.c Tue Apr 24 12:43:29 2012 (r234649) +++ user/andre/routelocking/net/route.c Tue Apr 24 12:54:04 2012 (r234650) @@ -425,6 +425,55 @@ done: return (newrt); } +int +rtlookup_fib(struct rtlookup *rtl, u_int fibnum, int flags) +{ + struct radix_node_head *rnh; + struct radix_node *rn; + struct rtentry *rt; + struct rm_priotracker tracker; + + KASSERT((fibnum < rt_numfibs), + ("%s: bad fibnum", __func__)); + + switch (rtl->rtl_dst->sa_family) { + case AF_INET: + case AF_INET6: + break; + default: + fibnum = 0; + } + + rnh = rt_tables_get_rnh(fibnum, rtl->rtl_dst->sa_family); + if (rnh == NULL) { + V_rtstat.rts_unreach++; + return (ENETUNREACH); + } + + /* Look up the address in the table for that Address Family. */ + RADIX_NODE_HEAD_RLOCK(rnh, &tracker); + rn = rnh->rnh_matchaddr(rtl->rtl_dst, rnh); + if (rn == NULL || (rn->rn_flags & RNF_ROOT)) + return (ENETUNREACH); + + rt = RNTORT(rn); + if (rt->rt_flags & (RTF_REJECT | RTF_BLACKHOLE)) + return (ENETUNREACH); + + /* Only copy DST when a gateway, otherwise route to interface. */ + if (rtl->rtl_gw != NULL && rt->rt_flags & RTF_GATEWAY) + bcopy(rt->rt_gateway, rtl->rtl_gw, SA_SIZE(rt->rt_gateway)); + rtl->rtl_ifp = rt->rt_ifp; + rtl->rtl_ifa = rt->rt_ifa; + rtl->rtl_mtu = rt->rt_rmx.rmx_mtu; + rtl->rtl_flags = rt->rt_flags; + if (flags & RTL_PKSENT) + rt->rt_rmx.rmx_pksent++; /* racy but ok - XXX WHY?*/ + RADIX_NODE_HEAD_RUNLOCK(rnh, &tracker); + + return (0); +} + /* * Remove a reference count from an rtentry. * If the count gets low enough, take it out of the routing table Modified: user/andre/routelocking/net/route.h ============================================================================== --- user/andre/routelocking/net/route.h Tue Apr 24 12:43:29 2012 (r234649) +++ user/andre/routelocking/net/route.h Tue Apr 24 12:54:04 2012 (r234650) @@ -83,6 +83,20 @@ struct rt_metrics { }; /* + * Pointers to structures on the stack for pure routing + * table lookups. + */ +struct rtlookup { + struct sockaddr *rtl_dst; /* Request */ + struct sockaddr *rtl_gw; /* Answer */ + struct ifnet *rtl_ifp; /* Answer */ + struct ifaddr *rtl_ifa; /* Answer */ + u_long rtl_mtu; /* Answer */ + int rtl_flags; /* Answer */ +}; +#define RTL_PKSENT 0x0001 /* increment packet sent counter */ + +/* * rmx_rtt and rmx_rttvar are stored as microseconds; * RTTTOPRHZ(rtt) converts to a value suitable for use * by a protocol slowtimo counter. @@ -120,13 +134,13 @@ struct rtentry { */ #define rt_key(r) (*((struct sockaddr **)(&(r)->rt_nodes->rn_key))) #define rt_mask(r) (*((struct sockaddr **)(&(r)->rt_nodes->rn_mask))) - struct sockaddr *rt_gateway; /* value */ int rt_flags; /* up/down?, host/net */ - int rt_refcnt; /* # held references */ + struct sockaddr *rt_gateway; /* the answer: nexthop to use */ struct ifnet *rt_ifp; /* the answer: interface to use */ struct ifaddr *rt_ifa; /* the answer: interface address to use */ struct rt_metrics_lite rt_rmx; /* metrics used by rx'ing protocols */ u_int rt_fibnum; /* which FIB */ + int rt_refcnt; /* # held references */ #ifdef _KERNEL /* XXX ugly, user apps use this definition but don't have a mtx def */ struct mtx rt_mtx; /* mutex for routing entry */ @@ -399,6 +413,7 @@ void rtalloc_ign_fib(struct route *ro, void rtalloc_fib(struct route *ro, u_int fibnum); struct rtentry *rtalloc1_fib(struct sockaddr *, int, u_long, u_int); int rtioctl_fib(u_long, caddr_t, u_int); +int rtlookup_fib(struct rtlookup *, u_int, int); void rtredirect_fib(struct sockaddr *, struct sockaddr *, struct sockaddr *, int, struct sockaddr *, u_int); int rtrequest_fib(int, struct sockaddr *, Modified: user/andre/routelocking/netinet/ip_fastfwd.c ============================================================================== --- user/andre/routelocking/netinet/ip_fastfwd.c Tue Apr 24 12:43:29 2012 (r234649) +++ user/andre/routelocking/netinet/ip_fastfwd.c Tue Apr 24 12:54:04 2012 (r234650) @@ -112,40 +112,22 @@ static VNET_DEFINE(int, ipfastforward_ac SYSCTL_VNET_INT(_net_inet_ip, OID_AUTO, fastforwarding, CTLFLAG_RW, &VNET_NAME(ipfastforward_active), 0, "Enable fast IP forwarding"); -static struct sockaddr_in * -ip_findroute(struct route *ro, struct in_addr dest, struct mbuf *m) +static int +ip_findroute(struct rtlookup *rtl, struct mbuf *m) { - struct sockaddr_in *dst; - struct rtentry *rt; /* * Find route to destination. - */ - bzero(ro, sizeof(*ro)); - dst = (struct sockaddr_in *)&ro->ro_dst; - dst->sin_family = AF_INET; - dst->sin_len = sizeof(*dst); - dst->sin_addr.s_addr = dest.s_addr; - in_rtalloc_ign(ro, 0, M_GETFIB(m)); - - /* * Route there and interface still up? */ - rt = ro->ro_rt; - if (rt && (rt->rt_flags & RTF_UP) && - (rt->rt_ifp->if_flags & IFF_UP) && - (rt->rt_ifp->if_drv_flags & IFF_DRV_RUNNING)) { - if (rt->rt_flags & RTF_GATEWAY) - dst = (struct sockaddr_in *)rt->rt_gateway; - } else { + if (rtlookup_fib(rtl, M_GETFIB(m), 0) || + !(rtl->rtl_flags & RTF_GATEWAY)) { IPSTAT_INC(ips_noroute); IPSTAT_INC(ips_cantforward); - if (rt) - RTFREE(rt); icmp_error(m, ICMP_UNREACH, ICMP_UNREACH_HOST, 0, 0); - return NULL; + return (0); } - return dst; + return (1); } /* @@ -160,13 +142,13 @@ ip_fastforward(struct mbuf *m) { struct ip *ip; struct mbuf *m0 = NULL; - struct route ro; - struct sockaddr_in *dst = NULL; + struct sockaddr_in dst; struct ifnet *ifp; struct in_addr odest, dest; u_short sum, ip_len; int error = 0; int hlen, mtu; + struct rtlookup rtl; #ifdef IPFIREWALL_FORWARD struct m_tag *fwd_tag; #endif @@ -180,7 +162,7 @@ ip_fastforward(struct mbuf *m) M_ASSERTVALID(m); M_ASSERTPKTHDR(m); - bzero(&ro, sizeof(ro)); + bzero(&dst, sizeof(dst)); /* * Step 1: check for packet drop conditions (and sanity checks) @@ -420,16 +402,23 @@ passin: /* * Find route to destination. */ - if ((dst = ip_findroute(&ro, dest, m)) == NULL) + bzero(&dst, sizeof(dst)); + dst.sin_family = AF_INET; + dst.sin_len = sizeof(dst); + dst.sin_addr.s_addr = dest.s_addr; + + bzero(&rtl, sizeof(rtl)); + rtl.rtl_dst = (struct sockaddr *)&dst; + if (ip_findroute(&rtl, m) != 1) return NULL; /* icmp unreach already sent */ - ifp = ro.ro_rt->rt_ifp; + ifp = rtl.rtl_ifp; /* * Immediately drop blackholed traffic, and directed broadcasts * for either the all-ones or all-zero subnet addresses on * locally attached networks. */ - if ((ro.ro_rt->rt_flags & (RTF_BLACKHOLE|RTF_BROADCAST)) != 0) + if ((rtl.rtl_flags & (RTF_BLACKHOLE|RTF_BROADCAST)) != 0) goto drop; /* @@ -476,8 +465,6 @@ forwardlocal: * "ours"-label. */ m->m_flags |= M_FASTFWD_OURS; - if (ro.ro_rt) - RTFREE(ro.ro_rt); return m; } /* @@ -490,10 +477,10 @@ forwardlocal: m_tag_delete(m, fwd_tag); } #endif /* IPFIREWALL_FORWARD */ - RTFREE(ro.ro_rt); - if ((dst = ip_findroute(&ro, dest, m)) == NULL) + dst.sin_addr.s_addr = dest.s_addr; + if (ip_findroute(&rtl, m) != 1) return NULL; /* icmp unreach already sent */ - ifp = ro.ro_rt->rt_ifp; + ifp = rtl.rtl_ifp; } passout: @@ -504,9 +491,7 @@ passout: /* * Check if route is dampned (when ARP is unable to resolve) */ - if ((ro.ro_rt->rt_flags & RTF_REJECT) && - (ro.ro_rt->rt_rmx.rmx_expire == 0 || - time_uptime < ro.ro_rt->rt_rmx.rmx_expire)) { + if (rtl.rtl_flags & RTF_REJECT) { icmp_error(m, ICMP_UNREACH, ICMP_UNREACH_HOST, 0, 0); goto consumed; } @@ -514,6 +499,7 @@ passout: #ifndef ALTQ /* * Check if there is enough space in the interface queue + * XXXAO: ifq access not locked and could be outdated! */ if ((ifp->if_snd.ifq_len + ip->ip_len / ifp->if_mtu + 1) >= ifp->if_snd.ifq_maxlen) { @@ -534,8 +520,8 @@ passout: /* * Check if packet fits MTU or if hardware will fragment for us */ - if (ro.ro_rt->rt_rmx.rmx_mtu) - mtu = min(ro.ro_rt->rt_rmx.rmx_mtu, ifp->if_mtu); + if (rtl.rtl_mtu > 0) + mtu = min(rtl.rtl_mtu, ifp->if_mtu); else mtu = ifp->if_mtu; @@ -550,7 +536,7 @@ passout: * Send off the packet via outgoing interface */ error = (*ifp->if_output)(ifp, m, - (struct sockaddr *)dst, &ro); + (struct sockaddr *)&dst, NULL); } else { /* * Handle EMSGSIZE with icmp reply needfrag for TCP MTU discovery @@ -583,7 +569,7 @@ passout: m->m_nextpkt = NULL; error = (*ifp->if_output)(ifp, m, - (struct sockaddr *)dst, &ro); + (struct sockaddr *)&dst, NULL); if (error) break; } while ((m = m0) != NULL); @@ -601,17 +587,13 @@ passout: if (error != 0) IPSTAT_INC(ips_odropped); else { - ro.ro_rt->rt_rmx.rmx_pksent++; IPSTAT_INC(ips_forward); IPSTAT_INC(ips_fastforward); } consumed: - RTFREE(ro.ro_rt); return NULL; drop: if (m) m_freem(m); - if (ro.ro_rt) - RTFREE(ro.ro_rt); return NULL; } Modified: user/andre/routelocking/netinet/ip_output.c ============================================================================== --- user/andre/routelocking/netinet/ip_output.c Tue Apr 24 12:43:29 2012 (r234649) +++ user/andre/routelocking/netinet/ip_output.c Tue Apr 24 12:54:04 2012 (r234650) @@ -119,12 +119,11 @@ ip_output(struct mbuf *m, struct mbuf *o int mtu; int n; /* scratchpad */ int error = 0; - int nortfree = 0; struct sockaddr_in *dst; + struct sockaddr_in dstn; struct in_ifaddr *ia = NULL; int isbroadcast, sw_csum; - struct route iproute; - struct rtentry *rte; /* cache for ro->ro_rt */ + struct rtlookup rtl; struct in_addr odst; #ifdef IPFIREWALL_FORWARD struct m_tag *fwd_tag = NULL; @@ -143,28 +142,9 @@ ip_output(struct mbuf *m, struct mbuf *o } } - if (ro == NULL) { - ro = &iproute; - bzero(ro, sizeof (*ro)); - -#ifdef FLOWTABLE - { - struct flentry *fle; - - /* - * The flow table returns route entries valid for up to 30 - * seconds; we rely on the remainder of ip_output() taking no - * longer than that long for the stability of ro_rt. The - * flow ID assignment must have happened before this point. - */ - if ((fle = flowtable_lookup_mbuf(V_ip_ft, m, AF_INET)) != NULL) { - flow_to_route(fle, ro); - nortfree = 1; - } - } -#endif - } - + /* + * Insert IP options. + */ if (opt) { int len = 0; m = ip_insertoptions(m, opt, &len); @@ -194,36 +174,20 @@ ip_output(struct mbuf *m, struct mbuf *o hlen = ip->ip_hl << 2; } - dst = (struct sockaddr_in *)&ro->ro_dst; -again: - /* - * If there is a cached route, - * check that it is to the same destination - * and is still up. If not, free it and try again. - * The address family should also be checked in case of sharing the - * cache with IPv6. - */ - rte = ro->ro_rt; - if (rte && ((rte->rt_flags & RTF_UP) == 0 || - rte->rt_ifp == NULL || - !RT_LINK_IS_UP(rte->rt_ifp) || - dst->sin_family != AF_INET || - dst->sin_addr.s_addr != ip->ip_dst.s_addr)) { - if (!nortfree) - RTFREE(rte); - rte = ro->ro_rt = (struct rtentry *)NULL; - ro->ro_lle = (struct llentry *)NULL; - } -#ifdef IPFIREWALL_FORWARD - if (rte == NULL && fwd_tag == NULL) { -#else - if (rte == NULL) { -#endif - bzero(dst, sizeof(*dst)); - dst->sin_family = AF_INET; - dst->sin_len = sizeof(*dst); - dst->sin_addr = ip->ip_dst; + if (ro != NULL) { + /* XXXAO: May use dst for nexthop. */ + if (ro->ro_rt != NULL) + RTFREE(ro->ro_rt); + ro = NULL; } + bzero(&rtl, sizeof(rtl)); +again: + dst = &dstn; + bzero(dst, sizeof(*dst)); + dst->sin_family = AF_INET; + dst->sin_len = sizeof(*dst); + dst->sin_addr = ip->ip_dst; + /* * If routing to interface only, short circuit routing lookup. * The use of an all-ones broadcast address implies this; an @@ -263,24 +227,13 @@ again: isbroadcast = 0; /* fool gcc */ } else { /* - * We want to do any cloning requested by the link layer, - * as this is probably required in all cases for correct - * operation (as it is for ARP). + * Look up the route to the destination. */ - if (rte == NULL) { -#ifdef RADIX_MPATH - rtalloc_mpath_fib(ro, - ntohl(ip->ip_src.s_addr ^ ip->ip_dst.s_addr), - inp ? inp->inp_inc.inc_fibnum : M_GETFIB(m)); -#else - in_rtalloc_ign(ro, 0, - inp ? inp->inp_inc.inc_fibnum : M_GETFIB(m)); -#endif - rte = ro->ro_rt; - } - if (rte == NULL || - rte->rt_ifp == NULL || - !RT_LINK_IS_UP(rte->rt_ifp)) { + rtl.rtl_dst = (struct sockaddr *)dst; + rtl.rtl_gw = (struct sockaddr *)dst; + rtl.rtl_ifp = ifp; + + if (rtlookup_fib(&rtl, inp ? inp->inp_inc.inc_fibnum : M_GETFIB(m), 0)) { #ifdef IPSEC /* * There is no route for this packet, but it is @@ -294,38 +247,29 @@ again: error = EHOSTUNREACH; goto bad; } - ia = ifatoia(rte->rt_ifa); + ifp = rtl.rtl_ifp; + ia = (struct in_ifaddr *)rtl.rtl_ifa; ifa_ref(&ia->ia_ifa); - ifp = rte->rt_ifp; - rte->rt_rmx.rmx_pksent++; - if (rte->rt_flags & RTF_GATEWAY) - dst = (struct sockaddr_in *)rte->rt_gateway; - if (rte->rt_flags & RTF_HOST) - isbroadcast = (rte->rt_flags & RTF_BROADCAST); + if (rtl.rtl_flags & RTF_HOST) + isbroadcast = (rtl.rtl_flags & RTF_BROADCAST); else isbroadcast = in_broadcast(dst->sin_addr, ifp); } + KASSERT(ifp != NULL, ("%s: ifp is NULL", __func__)); + /* * Calculate MTU. If we have a route that is up, use that, * otherwise use the interface's MTU. */ - if (rte != NULL && (rte->rt_flags & (RTF_UP|RTF_HOST))) { - /* - * This case can happen if the user changed the MTU - * of an interface after enabling IP on it. Because - * most netifs don't keep track of routes pointing to - * them, there is no way for one to update all its - * routes when the MTU is changed. - */ - if (rte->rt_rmx.rmx_mtu > ifp->if_mtu) - rte->rt_rmx.rmx_mtu = ifp->if_mtu; - mtu = rte->rt_rmx.rmx_mtu; - } else { + if (rtl.rtl_mtu > 0) + mtu = min(rtl.rtl_mtu, ifp->if_mtu); + else mtu = ifp->if_mtu; - } + /* Catch a possible divide by zero later. */ - KASSERT(mtu > 0, ("%s: mtu %d <= 0, rte=%p (rt_flags=0x%08x) ifp=%p", - __func__, mtu, rte, (rte != NULL) ? rte->rt_flags : 0, ifp)); + KASSERT(mtu > 0, ("%s: mtu %d <= 0, rtl=%p (rtl_flags=0x%08x) ifp=%p", + __func__, mtu, &rtl, rtl.rtl_flags, ifp)); + if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr))) { m->m_flags |= M_MCAST; /* @@ -333,7 +277,7 @@ again: * still points to the address in "ro". (It may have been * changed to point to a gateway address, above.) */ - dst = (struct sockaddr_in *)&ro->ro_dst; + //dst = (struct sockaddr_in *)&ro->ro_dst; /* * See if the caller provided any multicast options */ @@ -546,12 +490,12 @@ sendit: CSUM_DATA_VALID | CSUM_PSEUDO_HDR; m->m_pkthdr.csum_data = 0xffff; } + m->m_pkthdr.csum_flags |= + CSUM_IP_CHECKED | CSUM_IP_VALID; #ifdef SCTP if (m->m_pkthdr.csum_flags & CSUM_SCTP) m->m_pkthdr.csum_flags |= CSUM_SCTP_VALID; #endif - m->m_pkthdr.csum_flags |= - CSUM_IP_CHECKED | CSUM_IP_VALID; error = netisr_queue(NETISR_IP, m); goto done; @@ -559,7 +503,6 @@ sendit: /* Or forward to some other address? */ fwd_tag = m_tag_find(m, PACKET_TAG_IPFORWARD, NULL); if (fwd_tag) { - dst = (struct sockaddr_in *)&ro->ro_dst; bcopy((fwd_tag+1), dst, sizeof(struct sockaddr_in)); m->m_flags |= M_SKIP_FIREWALL; m_tag_delete(m, fwd_tag); @@ -629,7 +572,7 @@ passout: */ m->m_flags &= ~(M_PROTOFLAGS); error = (*ifp->if_output)(ifp, m, - (struct sockaddr *)dst, ro); + (struct sockaddr *)dst, NULL); goto done; } @@ -649,7 +592,7 @@ passout: goto bad; for (; m; m = m0) { m0 = m->m_nextpkt; - m->m_nextpkt = 0; + m->m_nextpkt = NULL; if (error == 0) { /* Record statistics for this interface address. */ if (ia != NULL) { @@ -663,7 +606,7 @@ passout: m->m_flags &= ~(M_PROTOFLAGS); error = (*ifp->if_output)(ifp, m, - (struct sockaddr *)dst, ro); + (struct sockaddr *)dst, NULL); } else m_freem(m); } @@ -672,9 +615,6 @@ passout: IPSTAT_INC(ips_fragmented); done: - if (ro == &iproute && ro->ro_rt && !nortfree) { - RTFREE(ro->ro_rt); - } if (ia != NULL) ifa_free(&ia->ia_ifa); return (error); Modified: user/andre/routelocking/netinet/tcp_output.c ============================================================================== --- user/andre/routelocking/netinet/tcp_output.c Tue Apr 24 12:43:29 2012 (r234649) +++ user/andre/routelocking/netinet/tcp_output.c Tue Apr 24 12:54:04 2012 (r234650) @@ -1232,7 +1232,7 @@ timer: ip->ip_off |= IP_DF; error = ip_output(m, tp->t_inpcb->inp_options, NULL, - ((so->so_options & SO_DONTROUTE) ? IP_ROUTETOIF : 0), 0, + ((so->so_options & SO_DONTROUTE) ? IP_ROUTETOIF : 0), NULL, tp->t_inpcb); } #endif /* INET */ From owner-svn-src-user@FreeBSD.ORG Tue Apr 24 16:43:53 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B4DE106566B; Tue, 24 Apr 2012 16:43:53 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 91F748FC0A; Tue, 24 Apr 2012 16:43:52 +0000 (UTC) Received: by lbbgm6 with SMTP id gm6so858012lbb.13 for ; Tue, 24 Apr 2012 09:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=x6jLqfXUCIY6WSn1MClNGdq8sVDmyaKRUih7Z8CAPaM=; b=aSoQnhYqx21auTiE7bi7qsWN30Ru90q1l0zkG8XMwehsfcmB7LBWvq2UFXXsMYpoTN sShXewUqOvBQ+gFPABEGHgMWsVIQXHb87thck7Wo0qkn1NBg4ZhpkWm5v6w5tVUWcKtV WJzx4rhsJoC9UpMU/T9ZXfnCU7RL+33i47RrktFU98CerCyVPsCaluI6OUS/hsekDqNx TBpOH8g5EdnaR7AH51fZIq3/0Fass3Wfb4kW5Uc4vjDGAkMZI/PrZTirAMBkgNB++7aB 4Evr3Q94O59hgA17cz7OtGVkffImp0KaBsTdGJTTuE25MOMS7Q1WA2IIes7BGoKimNoC w63g== MIME-Version: 1.0 Received: by 10.112.37.132 with SMTP id y4mr10130158lbj.8.1335285831234; Tue, 24 Apr 2012 09:43:51 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.63.146 with HTTP; Tue, 24 Apr 2012 09:43:51 -0700 (PDT) In-Reply-To: <201204241243.q3OChUqE048963@svn.freebsd.org> References: <201204241243.q3OChUqE048963@svn.freebsd.org> Date: Tue, 24 Apr 2012 17:43:51 +0100 X-Google-Sender-Auth: 9WXxnwuzKDzJ8l823mtbdNc7O64 Message-ID: From: Attilio Rao To: Andre Oppermann Content-Type: text/plain; charset=UTF-8 Cc: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: Re: svn commit: r234649 - in user/andre/routelocking: contrib/ipfilter/netinet contrib/pf/net kern net netinet netinet6 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 16:43:53 -0000 2012/4/24, Andre Oppermann : > Author: andre > Date: Tue Apr 24 12:43:29 2012 > New Revision: 234649 > URL: http://svn.freebsd.org/changeset/base/234649 > > Log: > hange the radix head lock to an rmlock (read mostly lock). > > There is some header pollution going on because rmlock's are > not entirely abstracted and need per-CPU structures. > > A comment in _rmlock.h says this can be hidden if there were > per-cpu linker magic/support. I don't know if we have that > already. Yes, we do, it is the DPCPU support. Attilio -- Peace can only be achieved by understanding - A. Einstein