Date: Mon, 03 Dec 2007 13:12:55 -0500 From: "Aryeh M. Friedman" <aryeh.friedman@gmail.com> To: Ade Lovett <ade@FreeBSD.org> Cc: David Southwell <david@vizion2000.net>, freebsd-ports@freebsd.org Subject: Re: [RFC/P] Port System Re-Engineering Message-ID: <47544727.9030209@gmail.com> In-Reply-To: <ADB6E4A8-DDA2-4C0A-A5E1-EF9A45A53753@FreeBSD.org> References: <200712030308.32301.david@vizion2000.net> <3BC04A18-86CD-4A93-831A-691EBD8D4A43@FreeBSD.org> <47543109.3050303@gmail.com> <7957A7E2-3ADA-4BD5-934C-F10A1F08F72F@FreeBSD.org> <47544020.5090605@gmail.com> <ADB6E4A8-DDA2-4C0A-A5E1-EF9A45A53753@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ade Lovett wrote: > > On Dec 03, 2007, at 09:42 , Aryeh M. Friedman wrote: >> If the questions are flawed then point out where. If the >> general concept of a survey vs. user stories vs. what ever then >> state which you think is more productive. If your problem is the >> medium/forum the data is being gathered in see below. > > I'll say this one last time. > > It is not a question of whether the questions are flawed, it is the > communications medium in which they were posted. > > You sent the "survey" to a single mailing list. ports@FreeBSD.org Wrong I also sent it to -char, -current, -stable and -questions. > > This is, statistically speaking, a self-selecting and exceptionally > minor part of the community to which the survey is addressed, most > notably: > > 1. The survey will only be seen by people that currently use > FreeBSD (subset 1) Better to refine the questions with them first then broaden the sample size then to ask the wrong questions to a large sample. > 2. It will further only be seen by those that are aware of the > ports@FreeBSD.org mailing list and have taken the conscious > decision to subscribe to it (subset 2) See above > 3. Within the readership of a relatively high volume mailing list > (to wit, the aforementioned ports@FreeBSD.org), a significant > minority, at best, will even notice the email (subset 3) That is a problem with any type on-line announcement/post... I have been critized in the past for being overly colorful in my subjects/headlines so this is an attempt to avoid being dismissed for being a *README* type post. > 4. Further, within that subset, you are soliciting responses, > whereas folks in general are notoriously lazy, or simply believe > that responding to such a random survey is a waste of their time > and effort (subset 4) That is true for any self-selected sample and by definition all on-line surveys are self-selected. > 5. Taking the sub-sub-sub-subset of folks that use FreeBSD, are > subscribed to the ports@ mailing list, have read your survey, and > responded to it, you then have to do analysis on what percentage of > responses are incomplete, flamebait, inaccurate, or otherwise > flawed. I have about 20 responses in private email and only the ones you have seen in public are in this category (many of the private ones actually thank me for the courage to conduct such a survey) > > For example, compare the number of discrete folks that are, to one > extent or another, contributing to this thread, as a percentage of > total FreeBSD installed systems. Care to guess what that > percentage is? Unless it's significant, any results can be > massaged into anything at all. Without a statistically valid > sampling, it is simply not possible to make any conclusions with > whatever dataset exists as a result of said survey. That is why I am slowly widening the set of places the survey is posted gives it time to mature and weed out any obviously badly worded questions and/or pointless ones (like the one on having to decide between fixing the worst and breaking the best) - -- Aryeh M. Friedman FloSoft Systems Developer, not business, friendly http://www.flosoft-systems.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHVEcn358R5LPuPvsRAn6wAJ43QOEv5tOmkV91qpfrlPxIuZMBjQCgoZAl OJyd2LmyaiheJaZaNbOKpuE= =ckPd -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47544727.9030209>