Diederik Stapel and the frequency of scientific shenanigans

On August 27, two junior researchers working with the Dutch social psychologist Diederik Stapel at Tilburg University contacted a university administrator with suspicions that their senior colleague was using faked data.  As one of the worst forms of academic shenanigans that fall under the broad umbrella of "academic misconduct", an allegation of data fabrication was quite serious.  This is especially true because Diederik Stapel was in the early stages of a prolific scientific career; he served on the editorial board of six different academic journals and had received the 2007 "Early Career Award" from the International Society for Self and Identity (ISSI).  He had also published many articles that received generous press attention, including one in Science that claimed that messy environments promote discrimination.

Nonetheless, a little over a week and one university investigation later, Stapel admitted to making up data and was sacked from Tilburg University.

The quest for (social) scientific truth

Every now and then when I tell a stranger that I study scientific psychology, I get a reaction that is perhaps best summarized by this comic from xkcd:

Click for a larger image

Essentially, the reaction is that, as a social science, psychology isn't a "real" science like biology, chemistry, or physics.  I encounter this reaction often enough, even among otherwise scientifically savvy people, that I am often tempted to do silly things like shout, punch things, or at the very least launch into an ill-advised rant.

A glimpse into the abyss of psychology prelims

Over the past month, I endured the crucible of the so-called "preliminary exams", or as they are more affectionately called, "prelims".  These exams go by different names in different areas ("qualifying exams" or "quals", "comprehensive exams" or "comps"), but across institutions, the intent is the same: complete an exam (or more rarely, write a paper) to prove your mastery of a body of knowledge.  Following prelims, graduate students are allowed to begin their dissertation research and, eventually, their PhD.

Needless to say, taking prelims is an intense and exhausting process.  While the specifics vary from place to place, it usually involves studying for months, followed by a multi-day exam with a strict deadline.  My own prelims consisted of a five-hour in-class test, followed by a six-day period in which I wrote four six-page essays.  I studied for my own exams for around five months and, according to my prelims notebook, I read some 75 papers and book chapters.  By the end of prelims I felt like I was leaking social psychology out the ears.

Six graphs answer questions about the PhD labor market

In my experience, getting an honest, straightforward answer about the post-PhD labor market from most professors in graduate school is about as easy as extracting teeth from the mouth of a sparrow.  Even when an answer is forthcoming, it is too often clouded by unrealistic expectations about the types of careers graduate students want and / or attempts to boost morale in order to increase research productivity.

Fortunately, no less an authority than the National Science Foundation has been conducting rigorous, nationally representative surveys on the US PhD labor market since 1993.  On top of that, the NSF has made the results from its biannual surveys open to the public, both in the form of raw data and in the form of summary statistics.  For a quantitative geek like me, the data are a little slice of heaven.

Thus, both to satisfy my own curiosity and for the benefit of other people who want hard data about PhD labor outcomes, I created the following six graphs with the goal of answering common questions about the STEM PhD labor market.  I will structure my graphs around three questions in particular:

(1) How successful are PhDs at finding the jobs they want?
(2) Where do PhDs go for their jobs and what do they do on the job?
(3) How well compensated are PhDs for the work they do?

Unraveling the "obviousness" bias in psychology

In my last post, I argued that the pseudoscience of parapsychology (and in particular the publication of a paper in the Journal of Personality and Social Psychology claiming to provide evidence of precognition) hurts the perception of psychology as a science. This may seem like an obvious argument to make; Bem's paper was published in the flagship journal of social psychology, so it is easy to make the logical jump that this article is representative of the kind of research most social psychologists do. Therefore, the reasoning goes, social psychology is not a real science at all; to quote one comment on the media coverage of the Bem article,

Psychology is such a joke. A demonstration of future events influencing present events would be one of the most important (if not *the* most important) findings in the history of mankind. Yet this demonstration doesn't end up in Science or Nature, but is published in the Journal of Personality and Social Psychology? And some wonder why psychology is still considered pseudoscience....

"Evidence" for precognition hurts the perception of psychology as a science

There is a a paper in press at the most prestigious journal in my field, Journal of Personality and Social Psychology, that claims to have found evidence of ESP.

In this paper, Daryl Bem, a well-known and well-respected psychologist at Cornell University, conducted 9 time-reversed versions of classic social psychology experiments. For example, one common way of testing for associations between sets of concepts is through priming, in which an picture is flashed very quickly on a computer screen, after which the participant must categorize a second object. People are usually faster at categorizing the second object when the two objects share a relationship to each other; for example, when a picture of a rainbow (which most people agree is "good") is displayed before the participant must categorize the word "puppy" (which most people also agree is "good") as either good or bad.