From owner-freebsd-performance@FreeBSD.ORG Thu Jan 6 05:22:47 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 769FC16A4CE for ; Thu, 6 Jan 2005 05:22:47 +0000 (GMT) Received: from web53902.mail.yahoo.com (web53902.mail.yahoo.com [206.190.36.212]) by mx1.FreeBSD.org (Postfix) with SMTP id 049C043D3F for ; Thu, 6 Jan 2005 05:22:47 +0000 (GMT) (envelope-from stheg_olloydson@yahoo.com) Received: (qmail 47982 invoked by uid 60001); 6 Jan 2005 05:22:46 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=Qj4fnXdtzh3Fyw7kYD4A9/rc+pyhlNlWzji2ayJkW/1UecJBnheW31Wp3AzBISMnvsNhHzbmzjhCfej5g38SLlo6EsVfFTxZzXgtrrfXn3RFyW0qyxdCsXdYFISo8Aa7iZvrfNSkocOpc6D2Vn7NN7xwpLyo30Fxi3pXhGiWWsA= ; Message-ID: <20050106052246.47980.qmail@web53902.mail.yahoo.com> Received: from [68.18.48.71] by web53902.mail.yahoo.com via HTTP; Wed, 05 Jan 2005 21:22:46 PST Date: Wed, 5 Jan 2005 21:22:46 -0800 (PST) From: stheg olloydson To: hubert@feyrer.de, g.mcgarry@ieee.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Thu, 06 Jan 2005 14:31:35 +0000 cc: freebsd-performance@FreeBSD.org Subject: Re: Benchmark: NetBSD 2.0 beats FreeBSD 5.3 in server performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2005 05:22:47 -0000 it was said: Gregory McGarry asked me to host and advertize this paper for him: Abstract: ``With the recent releases of NetBSD 2.0 and FreeBSD 5.3 operating system, many new and exciting features have been implemented. Both criticism and commendation on performance, reliability and scalability have been directed towards these releases. This paper presents a suite of benchmarks and results for comparing the performance of these operating systems. The benchmarks target core operating system functionality, server scalability and thread implementation. URL: http://www.feyrer.de/NetBSD/gmcgarry/ --------------- Hello, I read the "paper" with which you trolled a couple of FreeBSD lists. Here, quoting the "paper" is what stands out to me: Abstract: "These benchmarks are useful server-based criteria for demanding applications such as loaded webservers, databases, and voice-over-IP (VoIP) media relays. The results indicate that NetBSD has surpassed FreeBSD in performance on nearly every benchmark and is poised to grab the title of the best operating system for the server environment." System-call overhead "The results in Table 1 shows that NetBSD 2.0 marginally out-performs FreeBSD 5.3." Context-switch time "The results in Table 1 shows that NetBSD 2.0 marginally outperforms FreeBSD 5.3." Process lifecycle "The results in Table 1 indicate that NetBSD 2.0 marginally outperforms FreeBSD 5.3." Forking new processes "The FreeBSD kernel has an access time which scales linearly with the number of system processes. There are also many occasions when the access-time is very fast, resembling a constant access time." Memory-mapped files "The total of both benchmarks indicate that for a single mapping and subsequent access, FreeBSD shows a 38% performance improvement over NetBSD." Socket creation scalability "Neither NetBSD nor FreeBSD shows scalability problems." Binding addresses to sockets "This result indicates the FreeBSD scales better for binding addresses to sockets." Thread creation benchmark "For less than 250 threads, the time to create a thread is better in NetBSD than FreeBSD. For more than 250 threads, the thread creation time increases as the number of threads increases. Of particular concern, the relationship is not linear for the number of threads. Although one thousand threads is ample for most multi-threaded applications, the poor scalability may be a problem for some applications." Thread lifecycle, condition variables and mutexes "The results of these benchmarks for the basic POSIX thread primitives clearly shows that the NetBSD thread implementation contains many impressive optimizations." Thread context-switch time "The creation of the first thread incurs a significant penalty on NetBSD. However, the time to complete the game is significantly higher for FreeBSD over NetBSD. This is due to increased latency in the thread lifecycle and the much longer mutex acquire time for FreeBSD over NetBSD." Conclusions "Significant performance improvements are obviously visible in the thread implementation....Microbenchmarks are not always the best indicators to make judgments on the overall performance of one operating system over another...For many applications, the results presented in the paper may never affect performance. For others, the scalability of the operating system may simply not permit the application to run suitably." Clearly, the claims in the Abstract are not supported by the tests. The author even points out the inapplicability of the benchmarks that he ran. But even if they were, the author points out that only in the case of threading performance does NetBSD do appreciably better than FreeBSD. In all other tests, FreeBSD either does better, as well as, or only "marginally" worse than NetBSD. So what this should have been called is, "Meaningless Tests Show NetBSD 2.0 is Finally About as Good as FreeBSD 5.3." I want the 20 minutes that I spent reading this "paper" and responding to it back. If you are unwilling or unable to return the piece of my life you caused me to waste, I demand to personally apologise to everyone who reads not only the post, but the "paper", as well. stheg __________________________________ Do you Yahoo!? Yahoo! Mail - Easier than ever with enhanced search. Learn more. http://info.mail.yahoo.com/mail_250