Date: Mon, 26 Aug 2002 12:59:15 -0400 From: Ju Ichi <freebsd-security@ichi.net> To: "Sam Leffler (at Usenix)" <sam@usenix.org>, <freebsd-security@FreeBSD.ORG> Cc: "Nielsen" <nielsen@memberwebs.com> Subject: Re: IPSec SPD limit? Message-ID: <200208261259.15721.freebsd-security@ichi.net> In-Reply-To: <006101c24aff$cce8cd00$52557f42@errno.com> References: <200208231624.14487.freebsd-security@ichi.net> <006101c24aff$cce8cd00$52557f42@errno.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday 23 August 2002 7:49 pm, Sam Leffler (at Usenix) wrote: > > We are trying to setup a large IPSec SPD (in excess of 1000 SAs) on the > > following hardware/software config: > > > > Compaq DL360 with dual 1.4GHz processsors > > 2GB RAM > > 4GB swap space > > > > 4.6.1-RELEASE-p11 > > racoon-20020507a > > > > We get a "send: No buffer space available" when trying to read in the > > /etc/ipsec.conf file if it has more than about 1000 entries. Also, if we > do > > a setkey -DP after trying to read in /etc/ipsec.conf we get > > "recv: Resource temporarily unavailable" after it lists some of the SAs. > > > > Several kernel tweaks have been tried. For example, we have tried setting > > MAXUSERS from 0 to 1024 on bit boundaries (0, 128, 256, 512, and 1024). > > FWIW, setting it to 1024 seems to be evil. ;-) We have also tried > various > > settings in the kernel config file on NMBCLUSTERS, NMBUFS, NBUF, MAXDSIZ, > > MAXSSIZ, DFLDSIZ, and MAXFILES. In addition, we have tweaked > > kern.ipc.somaxconn, net.inet.tcp.sendspace, net.inet.tcp.recvspace, > > net.inet.udp.recvspace, and net.inet.udp.maxdgram after reading some > > performance tuning web pages. I can provide additional details as needed, > > but didn't want to make this initial request too long. > > > > Does anyone know of any limits on the number of entries the SPD can hold > and > > if so how to make the limits higher? > > > > setkey -DP returns the SA's via a PF_KEY socket. You're hitting the upper > bound on the amount of data that can be stored in a socket of this type as > all the data is returned en masse (i.e. the process isn't given an > opportunity to read data). PF_KEY sockets inherit the send+recv space > reserves of raw sockets: > > ebb# gdb -k /kernel /dev/mem > ... > (kgdb) p raw_sendspace > $1 = 8192 > (kgdb) p raw_recvspace > $2 = 8192 > > You can either work around this by upping these values or patch the PF_KEY > code to set the reserves on the socket explicitly (and provide sysctl's a la > udp+tcp to control the upper bounds). > > Sam Thanks! I changed /usr/src/sys/net/raw_cb.h as follows: < #define RAWSNDQ 8192 < #define RAWRCVQ 8192 --- > #define RAWSNDQ 65535 > #define RAWRCVQ 65535 So, now we have: (kgdb) p raw_sendspace $1 = 65535 (kgdb) p raw_recvspace $2 = 65535 We are able to get the policy loaded by using "setkey -c" with sleep statements as Nate suggested, but still are getting "recv: Resource temporarily unavailable" when doing a setkey -DP. Anymore ideas on other values to up? Also, Nate, do you know of a way to dump the poicy with setkey so it all shows? In other words, using setkey -c we can slow down the rate of putting entries in, but there doesn't seem to be a way to slow down the rate at which the policy is dumped. Thanks, Ju To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200208261259.15721.freebsd-security>