Date: Wed, 13 Aug 2008 16:12:39 +0200 From: Laszlo Nagy <gandalf@shopzeus.com> To: freebsd-questions@freebsd.org Subject: Max. number of opened files, efficiency Message-ID: <48A2EBD7.9000903@shopzeus.com>
next in thread | raw e-mail | index | archive | help
How many files can I open under FreeBSD, at the same time? Problem: I'm making a pivot table, and when I drill down the facts, I would like to create a new temporary file for each possible dimension value. In most cases, there will be less than 1000 dimension values. I tried to open 1000 temporary files and I could do so within one second. But how efficient is that? What happens when I open 1000 temporary files, and write data into them randomly, 10 million times. (avg. 10 000 write operations per file) Will this be handled efficiently by the OS? Is efficiency affected by the underlying filesystem? I also tried to create 10 000 temporary files, but performance dropped down. Example in Python: import tempfile import time N = 10000 start = time.time() files = [ tempfile.TemporaryFile() for i in range(N)] stop = time.time() print "created %s files/second" % ( int(N/(stop-start)) ) On my computer this program prints "3814 files/second" for N=1000, and "1561 files/second" for N=10000. Thanks, Laszlo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48A2EBD7.9000903>