Category Archives: Research methods

Doing Classroom-Based Research

Last Saturday, Anita Lämmerer and I had the privilege to facilitate a workshop in the ELT Connect 2015 conference that was  jointly organised by the Sprachausbildung and the Fachdidaktik sections of the Institut für Anglistik, University of Graz.

In our workshop, which was entitled Exploring Practice through Classroom-Based Research, we made the case for practitioner-led research, as a way for improving learning outcomes and driving professional development. During the workshop, the participants and we discussed the benefits of classroom-based research, and critically examined some assumptions that might inhibit or intimidate teachers who are considering such projects. We also engaged in a number of activities intended to exemplify how a classroom-based research project  might be planned.

We have uploaded (a modified version of) the slides and a copy of the worksheet we used during the workshop. We hope you might find them useful.

We have also uploaded a copy of a handout on Classroom-Based Research that we gave out to workshop participants. It is, by necessity, a very brief introduction to a vast topic, but we hope that it might provide some helpful orientation, if you are planning a research project, or if you are supervising or mentoring teachers who have to do such work.

We are very keen on reading any feedback you might want to share about the materials. We would also especially love to hear from you if they have inspired any classroom-based research projects.


If you’d like to get in touch, you can do so through the contact page in this website, or by sending us an email at Achillefs.Kostoulas@uni-graz.at (Yes, there’s an f in my university email – don’t ask!)

Featured Image: Pencils “All in a Row”, by JLS Photography – Alaska (CC BY-NC-ND-2.0

TESOL Research Agenda 2014

TESOL International recently published their 2014 research agenda, a document that aims to summarise salient theoretical developments and gaps in the profession’s collective knowledge, and provide guidance for new research projects.

Future research

In the document, three main domains of research are defined: (a) research on individuals engaged in language teaching and learning, (b) research on learning communities, such as classrooms, schools, or online networks, and (c) research on societal change as it relates to language learning and teaching.

Each of these domains, it is argued, are impacted by developments on three fronts. First, recent years have seen theoretical developments in our understanding of language, including recent views of language as “a continually emerging, socially mediated, and self-organizing resource for identity construction and interaction” (p. 8). Secondly, there has been increasing interest in the way technology can be used to support and assess learning. Finally, research has tended to be concerned with teacher agency. For many teaching practitioners, research is conceptualised “as not only answering questions about what to do but also helping to improve the ways they develop professionally” (as above). The connections between the three domains of research and the three developments are visually represented here (clicking on each domain will pull up a number of example research questions).

While this agenda seems, in some ways, rather conservative, it does seem to contain some early indications of a paradigm shift in language education. In my PhD thesis, I developed a framework for describing language education, which consisted of three aspects and three paradigms. This is presented below, in a slightly modified form:

Content (What?)

Method (How?)

Impact (Why?)

Technical

Standard language
(a “single monochrome strand”)

Transmissive
(e.g., grammar-translation)

Reproduction of status quo

Mainstream

Inclusive Target Language definition
(e.g., World Englishes)

Communicative
(e.g., CLT, TBL)

Awareness of diversity

Critical

Dynamic conceptualisations of language
(e.g., some strands of ELF)

Post-method pedagogy
(e.g., appropriate methodology)

Resistance of hegemony

Table 1 – A theoretical framework for describing language education (based on Kostoulas 2014

Using this framework, it would seem that the TESOL agenda aligns with critical conceptualisations of content, and -in addition- the research aims concerningagency seem to point towards the critical position of Resistance. It remains to be seen how the research aims outlined in the agenda will be implemented, and what impact they will have on the profession, but I have a definite sense that times are indeed a-changing.

Research Ethics

Another point of interest in the publication is the section on Research Ethics, which lists a number of questions that researchers need to be aware of, before, during and after their research (p. 6). I’ve copied these below (keeping the US spelling conventions of the original):

Before:
  • Who will benefit and how from the proposed research? Is this just?
  • What are potential risks from this research?
  • Has everything been done to minimize potential risks, especially to vulnerable populations?
  • Do the benefits warrant any risks associated with the proposed research?
  • What guidelines for ethical research already exist?
During
  • Do research participants understand the goals of the research and what is required of them?
  • Are they participating willingly and with full understanding of the research’s purpose and their own rights?
  • Have I minimized factors that might coerce them to participate?
  • What impact is participation having on those involved in the research?
  • Are any unexpected consequences occurring?
  • Are the interests and needs of the participants being prioritized over the goals of the research?
After
  • Are the findings of the research being disseminated in a way that most benefits society and the participants?
  • Is the confidentiality and security of data from the project being adequately monitored?
  • Are the interpretations and uses of research findings by others being monitored for appropriateness?

The publication also contains a list of references to other research ethics codes drafted by other scholarly societies (e.g., AERA, BAAL) which may be of interest to researchers (p.22). In light of the casual disregard of ethics that I have sometimes encountered, I think that such publications are useful reminders of how to conduct research responsibly.

~

The research agenda is a fairly concise document (22 pages, including front matter and appendices), and I strongly recommend that it is read by anyone who has an interest in language education research. To download the document in .pdf form, click on the link below:

TESOL International Association. (2014). TESOL Research Agenda 2014. Alexandria, VA: Author.


Featured image by The Leaf Project @ Flickr (CC BY-SA 2.0)

How to summarise Likert scale data using SPSS

Elsewhere in this blog, I have written that a Likert scale might consist of several overlapping items. For instance, if I want to measure subjects’ attitudes towards sweets, I might ask them to record how they feel about the following statements:

Strongly Agree Agree Disagree

Strongly Disagree

1.  I like chocolate.
2.  I like cookies
3.  Ι Iike whipped cream

In order to interpret these data, we need to summarise the data in the scale. The safest way to do this is by estimating the median value of all the items. Using the same example as above, I need to create a new ‘super-variable’, which shows the mean of items (1), (2) and (3) for each respondent.

In the paragraphs that follow, I will show how to do this, using SPSS. Continue reading How to summarise Likert scale data using SPSS

Dependent and Independent variables, using SPSS, and minding one’s manners

Every week, I receive between two and five emails asking about research questions, most of which I do not answer because I don’t have time, and because I have already answered them (or similar questions) repeatedly in this blog. When I do reply, it is usually because an email provides the affordance for a teaching point, and the message that follows provides no fewer than three:

  1. What is the difference between Dependent and Independent variables?
  2. Must we really use SPSS to do statistics?
  3. What are some good norms for requesting assistance?

Continue reading Dependent and Independent variables, using SPSS, and minding one’s manners

How not to do a research project

I hope I do not sound too immodest when I claim that I have some experience in school-based research. My PhD research was embedded in a language school, I’ve supervised BA and M-level research projects which involved working with schools, students and their parents, and I was for some time responsible for school-based research in a primary school affiliated to the University of Ioannina, in Greece.

But the post that follows does not draw on any of this expertise; rather, it’s told from my perspective as a parent. A few weeks ago, I was asked to consent to my daughter’s participation in a research project that would take place in her primary school, and I have since become a witness to what appears to be one of the most badly-conducted MA dissertation studies I have ever come across. What follows, therefore, serves the dual aim of venting some of my frustration, and offering some recommendations which may be of use in designing school-based research interventions.

Continue reading How not to do a research project