Idealized vs actual psychological science

I have been reading recently about the philosophy of science, which has got me thinking about the scientific method, both as it's taught in most psychology classes and as it's commonly practiced in psychology.  This thinking has led me to the following conclusion: the version of the scientific method that is usually taught in psychology classes is a farce, to the detriment of the science as a whole.  Let me explain.

Differences between psychology's ideal and actual scientific method

In most psychology classes, the scientific method is explained as follows:

Click to embiggen

This idealized version of science (which I will call confirmatory science) starts with a theory, which the scientist uses to derive predictions through deductive logic.  The scientist then figures out a method to test the theory-derived predictions.  Using the method, the scientist gathers data and compares the data to the predictions, usually using statistical analysis.  Based on the comparison of predictions to data, the scientist either increases his or her confidence in the theory or revises the theory.  Science complete.

The trouble is that, at least in my experience, psychological science usually doesn't work this way.  Instead, it works something like the following, which I will call actual science (remember, of course, I'm just making claims about psychology):

Click to embiggen

As in confirmatory science, actual science starts with a prediction.  However, the source of this prediction is not usually theory, but instead a hunch, real-world observation, clinical observation, or some other extra-theoretical means.

The scientist then develops a method to test the prediction, gathers data based on the method, and compares the data to the prediction, usually using statistical analysis.  If the data are consistent with the prediction, the scientist develops a theory that logically implies the original prediction as a consequence of its premises.

On the other hand, if the data are inconsistent with the prediction, the scientist has to figure out what went wrong.  Usually (but not always), the scientist concludes that the method was flawed rather than concluding that the prediction was incorrect.

What do we make of the discrepancies between these two versions of science?

First, it is useful to make the following observations about confirmatory science:

(1) Confirmatory science assumes that a strong theory already exists that can be used to derive one's predictions
(2) Confirmatory science views deductive logic as the "best" way of deriving predictions

(3) Confirmatory science assumes that the method used to test one's predictions is relatively free of error

Unfortunately, psychology has few pre-existing theories that are strong enough to serve as a logical basis to deduce predictions.  Moreover, most methods in psychology are quite error-prone, creating a logical ambiguity whereby it is difficult to know whether results that are inconsistent with predictions were obtained because the prediction was wrong or because of method error.

The difficulty of confirmatory science in psychology leads me to make the following claim:

On its own, confirmatory science is not a good  way of doing psychology

In fact, I believe the overemphasis on confirmatory science in psychology has these specific negative consequences:

(1) Confirmatory science discourages scientists from acknowledging the true sources of their ideas.  Based on my own experience, many ideas in psychology come from sources that are different from theory.  However, because psychology journals enshrine the deductive, confirmatory scientific method as the "best" method, scientists are encouraged to present their ideas as if they were derived from theory.  This is a farce.  Let's acknowledge the true sources of our ideas so that we can focus on building theories that might eventually serve as a better basis of prediction.

(2) Confirmatory science does not acknowledge the role of method error in psychology.  Our methods are studying people are relatively prone to error.  It is important for psychologists to explicitly recognize this fact so that we can properly interpret whether unexpected results are due to poor methods or false premises.

(3) Confirmatory science prioritizes making predictions in new data at the expense of explaining findings in old data.  Explaining findings in old data involves noticing patterns across several studies and developing theory that explains those patterns.  However, confirmatory science does not have an explicit place for theory development -- instead, it prioritizes using theory to logically derive predictions in new situations.

Addressing the non-cumulative character of psychology

In 1978, psychologist Paul Meehl made the following observation about theories in psychology:

Most of them [psychological theories] suffer the same fate that General MacArthur ascribed to old generals -- They never die, they just slowly fade away.

Meehl's observation is just as true now as it was in 1978.

There are many reasons for the non-cumulative character of psychology, and fully exploring these reasons would involve a completely new post.  However, I submit that one of the reasons is that psychologists are loathe to acknowledge how soft their theories truly are.

Confirmatory science is just one way of doing science, and it is only possible in the presence of strong prior theory.  Let's acknowledge that fact and devote our attention to explicitly developing these strong theories.  Until we do this, I fear that psychological theories are doomed irrelevance, just like MacArthur's old generals.


References

Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834. http://doi.org/10.1037/0022-006X.46.4.806