tag:blogger.com,1999:blog-5352450344824585503.post3050379104349196573..comments2023-10-20T02:20:45.487-05:00Comments on Patrick S. Forscher: Idealized vs actual psychological scienceUnknownnoreply@blogger.comBlogger2125tag:blogger.com,1999:blog-5352450344824585503.post-42581410512515277822017-03-30T11:42:23.716-05:002017-03-30T11:42:23.716-05:00I admit that the "Choosing predictions" ...I admit that the "Choosing predictions" paper has been on my reading list for some time! Maybe this will be a good excuse to finally get around to reading it. :)<br /><br />With regard to your first comment, I am defining "method error" to encompass both systematic and non-systematic error. Non-systematic error is less of a problem with large samples. Systematic method error (e.g., measures that are invalid because they do not tap the desired construct) are a huge problem even in unlimited samples.<br /><br />Another way of looking at the criticism that I tried to articulate in this post is that psychology has a huge problem with not developing complete models of whether and how the underlying constructs relate to the measures/methods we use to assess them. This is what creates ambiguity when our results do not conform to "theory": "theory" here could mean either the scientific theory that we ultimately want to address with our data or the various meta-theories about how our methods allow us to assess our underlying constructs.<br /><br />In other words, I'm not arguing against replication at all -- I am arguing that the lack of clear attention to methodological validity adds logical ambiguity that impedes scientific progress.<br /><br />See also this paper, "Attack of the Psychometricians" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2779444/). I didn't know about it when I wrote this post, but it makes similar points, I think.Patrick S. Forscherhttps://www.blogger.com/profile/05573437614920221191noreply@blogger.comtag:blogger.com,1999:blog-5352450344824585503.post-60641420837843975572017-03-27T05:32:58.881-05:002017-03-27T05:32:58.881-05:00To me, it seems that the points 2) and 3) might be...To me, it seems that the points 2) and 3) might be in disagreement with each other; if the data is noisy, should we be explaining findings before making sure they are not due to method error? This, among other things, would mean replication – i.e. predicting that the same pattern appears in new data, right? <br /><br />I'd be interested to hear your thoughts about the recent paper "Choosing prediction over explanation in psychology"! [pilab.psy.utexas.edu/publications/Yarkoni_Westfall_PPS_in_press.pdf]<br /><br />Anonymoushttps://www.blogger.com/profile/05343549436175255815noreply@blogger.com