Unfortunately, my limited understanding of the do's and don'ts of data analysis keeps me from going beyond such vague doubts, so my conservative response is to basically disregard such findings.
Of course, there's usually some "validation" thrown in the final report/paper to show that the statistical analysis is on the up-and-up, but the blatant publish-at-all-cost attitude behind it all leaves me doubtful. Here's the typical scenario: costly experiment gets carried out (without much thought given to the subsequent analysis), the original researchers cannot readily discern a "story" in the gathered data, someone gets brought in to apply some "statistical wizardry", and who, after slicing and dicing the data every which way, finally manages to extract some publishable "story" from it. In my line of work I often come across what looks to me like rampant "data snooping", or perhaps it would be better described as "data torture", though those doing it seem to see the same activity as entirely reasonable and unproblematic "exploration".
On the other hand, "exploratory data analysis" seems to be a perfectly respectable procedure in statistics, at least judging by the fact that a book with that title is still reverentially cited as a classic. Many times I have come across informal warnings against "data snooping" (here's one amusing example), and I think I have an intuitive idea of roughly what that means, and why it may be a problem.
0 kommentar(er)
