Date: Thu, 22 Dec 2011 10:00:19 -0800 From: <matthew@phoronix.com> To: "Adrian Chadd" <adrian@freebsd.org> Cc: Alexander Leidinger <Alexander@leidinger.net>, freebsd-performance@freebsd.org, Erik Cederstrand <erik@cederstrand.dk> Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server Message-ID: <20111222180020.6D422106566B@hub.freebsd.org> In-Reply-To: <CAJ-Vmo=jRwprjW%2BKCQ%2Bs7f2J-tFQvZbJ2CwkA_9rGTO5c6JkMw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A =0A =0A =0A =0A Let me suggest an alternative. =0A = =0A Within the Phoronix Test Suite ecosystem, we have a continious=0A = integration/validation system called Phoromatic=0A ([1]http://www.phoromatic.c= om/). We have a brief theory of operation=0A on it captured=0A= =0A [2]https://docs.google.com/present/view?id=3Dds439pg_42hpg57m86=0A . = You can see a Linux oriented tracker available at=0A [3]http://ph= oromatic.com/kernel-tracker.php (the nice graphs are when=0A you sel= ect 180 days and the ION 330 systems). We've re-assigned the=0A sy= stems to other projects so it is no longer being updated, but it=0A serv= ed our purposes. =0A =0A This system will allow you to just de= dicate a machine to be=0A updateable and pick up the directions for whic= h test to be run. The=0A FreeBSD project (or contributors) would m= aintain the slave or test=0A machines. The test suite would be sel= ected and managed by the=0A FreeBSD project or contributors. The g= lue code to emit triggers=0A (possibly SVN, git or other submission, or = even just daily) and the=0A scripts to update the systems would also be = maintained by the=0A FreeBSD community. =0A =0A Phoronix Me= dia would be happy to host it on Phoromatic.com (we've=0A played with ho= sted with project branding) and provide the data store=0A and the analyt= ics. We'd also be willing to make enhancements to=0A support the F= reeBSD project. =0A =0A This should solve the "I don't have ti= me to maintain a automated=0A test infrastructure". You don't need= to, just write the glue=0A scripts, and dedicate a couple of machines.&= nbsp; I believe FreeBSD=0A vendors like ixSystems may be able to support= this effort with a=0A dedicated machine. You can have as many mac= hines as you like=0A demonstrating AMD/Intel/32/64/large mem/low mem,etc= =2E =0A =0A The comments around interactivi= ty can also be measured to some=0A extent. We have the model of a = "monitor". This can be configured=0A to determine jitter around a = number of system variables and to=0A possibly inject actions to measure = impacts. =0A =0A We're more than happy to work with you guys, = and are willing to help=0A do a lot of the infrastructure and automation= lifting. =0A =0A Regards, =0A =0A Matthew =0A= =0A =0A =0A On 12/22/11 8:56 AM, Adrian Chadd wrot= e:=0A =0A Guys, gir= ls, fuzzy creatures,=0A=0AThis is by far the best example of a constructive= email in this entire thread.=0A=0AIf people would like to help, Erik here = is exactly the kind of person=0Awith exactly the kind of software that need= s a hand.=0A=0AI think enough philosophizing has been done - now we have qu= estions=0Athat need answering; theories that need testing. And that require= s,=0Ayou know, coding. :)=0A=0AThe best thing right now would be for *BSD p= eople to pick up the=0APhronix test suite, try to compile/run it, and provi= de feedback. Do=0Ayour own benchmarks on your own hardware and report back = the results.=0AThat's how we fix the "benchmarking problem." We don't fix i= t by=0Aarmchair philosophy, we fix it by getting our hands dirty. :)=0A=0A2= c,=0A=0A=0AAdrian=0A=0A=0AOn 22 December 2011 03:21, Erik Cederstrand [4]<erik@ = cederstrand.dk> wrote:=0A =0A = =0A Den 21/12/2011 kl. 19.48 skrev Alexander Leidinge= r:=0A=0A =0A =0A And related to the subject: wasn't it you who developed the automatic= benchmarking stuff? If yes, why not make it available? If you don't have h= e resources, I offer my help to make it available somewhere.=0A =0A = =0A Yes, that's me. I'm mostly out = of time right now, but I'd like to offer help if someone wants to pick up t= he project.=0A=0AFor those who haven't heard about it, it's a system design= ed specifically to track performance of FreeBSD over time by comparing revi= sions of FreeBSD, everything else being equal. It consists of a tinderbox-l= ike build script for a build server, a script to install FreeBSD and run be= nchmarks on at least one slave, and a database-backed website to aggregate = and visualize results.=0A=0AThe framework does work as-is, but it really ne= eds to be updated: convert the scripts to use the SVN repo instead of CVS, = improve visualization and search on the web fronted, and improve the benchm= arking script so it's easier to extend. I don't have hardware available to = run the benchmarks, but I think there's hardware available in the FreeBSD c= luster.=0A=0AHere's a link to the source code: [5]http://dev.affect-it.dk= /tracker.tgz=0AAnd to my thesis describing how it works: [6]http://de= v.affect-it.dk/tracker.pdf=0A=0AJust send me a mail if you're intereste= d.=0A=0AThanks,=0AErik_______________________________________________=0A[7]free bsd-performance@freebsd.org mailing list=0A[8]http://lists.freebsd.org/mailman/ listinfo/freebsd-performanc= e=0ATo unsubscribe, send any mail to [9]"freebsd-perfo= rmance-unsubscribe@freebsd.org"=0A =0A =0A = _______________________________________________=0A[10]freebsd-performance@freeb sd.org mailing list=0A[11]http://lists.freebsd.org/mailman/listinfo/freebsd-per formance= =0ATo unsubscribe, send any mail to [12]"freebsd-performanc= e-unsubscribe@freebsd.org"=0A =0A =0A =0A = =0A=0A=20 References 1. 3D"http://www.phoromatic.com/" 2. 3D"https://docs.google.com/present/view?id=3Dds439pg_42hpg57m86= 3. 3D"http://phoromatic.com/kernel-tracker.php" 4. 3D"mailto:erik@cederstrand.dk" 5. 3D"http://dev.affect-it.dk/tracker.tgz" 6. 3D"http://dev.affect-it.dk/tracker.pdf" 7. 3D"mailto:freebsd-performance@freeb= 8. 3D"http://lists.freebsd.org/mailman/listinfo/freebs= 9. 3D"mailto:freebsd-performance-unsubscribe@freebsd.org" 10. 3D"mailto:freebsd-performance@freebsd.or= 11. 3D"http://lists.freebsd.org/mailman/listinfo/freebsd-per= 12. =3D"mailto:freebsd-performance-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111222180020.6D422106566B>