Omitting the control group

I was recently asked the following question about experimental research, and I am copying my answer here, in case it is of interest to readers of the blog:

When can we omit or not use the control group in our research and we can use more than one experimental group instead? Sometimes we have to do this in Education. When and why?

It is indeed possible to do research with two or more experimental groups and no control group, but there are some caveats.

In education research, a study with more than one experimental groups would be appropriate when we want to compare the effects of variables with multiple values: e.g., syllabuses, teaching methods or approaches, etc. Provided that our sample is large enough, and group membership is assigned randomly, we can assume that all the experimental groups are more-or-less similar in most respects. If we then applied a different intervention with each group, we might deduce that any differences in learning outcomes reflect differences in the effectiveness of the two syllabuses, methods etc. that we are comparing.

Let me illustrate with an example: Suppose we are interested in improving fluency in a foreign language: it seems reasonable that people become more fluent by exposure to input, but we might want to find out whether it’s more effective to spend more time reading or listening, so that we can decide whether to invest in a new library or more audiovisual equipment. To find out more, we could assign students to two experimental groups, one of which would spend an hour a week watching TV programming in a foreign language, while the other would spend an hour a week reading books. By measuring the students’ fluency before and after the interventions, we’d be able to make a claim as to which method was more effective.

However, a research design without a control group comes with an important limitation: We cannot be certain that the outcomes were caused by our interventions. In the example above, it is possible that students became more fluent because they had more practice speaking at school; or because as they grew older they became more confident; or maybe even because they knew that they were being tested, and therefore practised more (a phenomenon known as the Hawthorne effect). So, while I might claim, on the basis of a comparative experiment, that watching television is more (or less) strongly associated with developing fluency than reading is, I cannot claim with equal confidence that “if you want to become more fluent you need to watch more television”.

So then, when should we use more than one experimental groups without a control? Ideally, you would use this design when the causal links between your interventions and their purported outcomes are well-established in the literature, and you are only interested in comparisons (i.e., you know that method A works, you also know that method B works, and you just want to find out which is better). This is something that you will have to establish in the literature review. If this is not the case, but you still cannot use a control group for pragmatic reasons (perhaps because you don’t have access to enough participants), then you would have to acknowledge the lack of a control as a limitation of the study when reporting your findings.

Further reading:

  1. Seliger, H. W., & Shohamy, E. (1989). Second language research methods. Oxford: Oxford University Press. pp. 140-141.
  2. Brown, J. D., & Rodgers, T. S. (2002). Doing second language research. Oxford: Oxford University Press. pp. 211-213
  3. Dörnyei, Z. (2007). Research methods in applied linguistics : quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press.pp. 116-117

One comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.