Peter Goodyear

It’s entirely my fault. I chose to subscribe to a daily email from Times Higher Education. No-one forced me. I could cancel at any time. It’s also my fault that I get snarky when they do one of these:

That’s from today’s THE email bulletin. The story to which it refers is behind a paywall, but my university subscription gives me access. If you don’t have access to the THE and your scepticism setting has been dialled back, you may find your brain registering the belief that

“teaching students online produces the same academic performance as face-to-face teaching”.

After all, that’s what you just read in the THE bulletin. If you are an inexperienced university manager, over-stretched educational policy maker, researcher for a free market think tank or EdTech entrepreneur, such a belief could prove unhelpful to the rest of us.

Turning to the story itself, in the THE, we are told:

“The results, which come at a time when universities across the world have had to shut their campuses and move to online instruction as a result of the coronavirus pandemic, found that students who were taught wholly online scored the highest on their average scores in assessments taken throughout the course. Those taught fully online scored, on average, 7.2 percentage points higher than the other two forms of learning.”

But if we look at the journal article on which this is based, we find that the researchers are quite clear about why the ‘fully online’ students did better. Unlike the students with whom they were being compared, the fully online students had three goes at each of the weekly assessment tasks. The students who were taught in ‘in-person’ or ‘blended’ modes did not have this advantage. The researchers describe this as “an artefact of the more lenient assessment submission policy for online students” (Chirikov et al., 2020, p2). In the journal article, this caveat comes immediately after the finding quoted above in the THE piece. Yet it didn’t make it into the THE or the THE’s email bulletin.

Casual readers of the email bulletin might also assume that the proposition about equivalent performance comes from a representative sample of comparisons between online and face-to-face teaching. They might be surprised to learn that the research was conducted in three Russian universities, in two courses: Engineering Mechanics and Construction Materials Technology. The online course instructors came from one of Russia’s “top engineering schools” while the instructors involved in the ‘in person’ and ‘blended’ modes worked in one of the students’ own universities. These are described in the journal article as ‘resource-constrained institutions’ and the researchers characterise the instructors in the ‘resource-constrained’ universities as having weaker educational backgrounds, fewer research publications and less teaching experience than the instructors from the ‘top engineering school’ (Chirikov et al., 2020, p2).

The THE article does not refer to any prior studies. The extensive review and meta-analysis work by Barbara Means and colleagues may as well not exist.

My point: I’m not criticising the researchers (Igor Chirikov and colleagues). They put a lot of care into this study and they are serious about improving access to STEM education opportunities. I just wonder how one of our ‘top trade papers’ (THE) can provide such bad service to its industry, and what we can do to help improve the situation.

References

Chirikov, I., Semenova, T., Maloshonok, N., Bettinger, E., & Kizilcec, R. F. (2020). Online education platforms scale college STEM instruction with equivalent learning outcomes at lower cost. Science Advances, 6(15), eaay5324. doi:10.1126/sciadv.aay5324

Means, B., Bakia, M., & Murphy, R. (2014). Learning online: what research tells us about whether, when and how. New York: Routledge.

Means, B., Toyama, Y., Murphy, R. F., & Baki, M. (2013). The effectiveness of online and blended learning: a meta-analysis of the empirical literature. Teachers College Record, 115, 1-47. 

Ellis, R., & Goodyear, P. (Eds.) (2018) Spaces of teaching and learning: integrating research and practice, Singapore: Springer. 243pp.

Markauskaite, L & Goodyear, P. (2017) Epistemic fluency and professional education: innovation, knowledgeable action and actionable knowledge, Dordrecht: Springer. 651pp.

Carvalho, L., Goodyear, P. & de Laat, M. (Eds.) (2017) Place-based spaces for networked learning, New York, RoutledgeFalmer, 288pp.

Carvalho, L. & Goodyear, P. (Eds.) (2014) The architecture of productive learning networks, New York, RoutledgeFalmer, 294pp.

Luckin, R., Puntambekar, S., Goodyear, P., Grabowski, B., Underwood, J. & Winters, N. eds. (2013) Handbook of design in educational technology, New York, Routledge, 509pp

Ellis, R., & Goodyear, P. (2010).
Students’ experiences of e-learning in higher education: the ecology of sustainable innovation. Abingdon: Routledge, 232pp

Goodyear, P., & Retalis, S. eds. (2010).
Technology-enhanced learning: design patterns and pattern languages.
Rotterdam: Sense Publishers, 330pp.

Over the last few years, I’ve been claiming that there is a huge amount of educational design knowledge embedded in the working practices of experienced university teachers. This knowledge is very unevenly distributed and we need better ways of sharing it.

With colleagues Lucila Carvalho, Kate Thompson, Pippa Yeoman and others, I’ve tried to promote some ways of working on this problem. Among them is the ‘ACAD’ framework, which is meant to help designers think separately about – and then bring into some kind of harmony – task design, social design and the design (or setting in place) of material and digital tools and resources. In other words, design needs to attend to (a) what students are being asked to do, (b) how they should work together to do it, (c) what tools etc they’ll need (with some careful thought about what can be digital, what should be in material form and so on).

All of this design thinking needs to be understood as non-deterministic: design works indirectly – what students actually do at ‘learn time’ is what shapes the actual outcomes of the task they tackle. But that dependence on what students actually do doesn’t absolve the teacher-as-designer of the responsibility for thinking things through carefully. Far from it.

I’ve been writing and giving talks about this for 20 years or so. Sometimes people get it. Sometimes I feel they don’t. The ACAD framework and some of the thinking behind it can be found in my other design papers on this site. There’s also a really good new paper in the British Educational Research Journal by Lucila and Pippa.

But just a few minutes ago I read this post by Danica Savonick on the hastac website and it is just fabulous: both as an example of the careful thinking that has gone into the design and (selfishly) as an illustration of what we keep banging on about with ACAD.

Please take 5 and read it. You don’t need to be a literature teacher. You just have to care about students learning.

And bye-the-bye, it’s a lovely illustration of what we talk about in our ACAD shtick.

Two key points:

1) You don’t need ACAD (or any formalised model of ‘how to do design for learning’) to come up with a design like the one Danica Savonick is sharing. I understand her example has emerged from her own practice and quite likely has evolved over a few trials. It’s what designers can do, without knowing they are designing or thinking of themselves as designers (or wearing black clothes). I see lots of academics solving very complex design problems without positioning themselves as designers or drawing on ‘how to design for learning’ texts or methods. NB in saying this, I’m not taking anything away from what Danica Savonick has designed. I don’t know her and for all I know she has some background in ID. (I just don’t think that’s the case though. The example reads like a pure distillation of knowledge accumulated in practice rather than anything inflected with justifications from learning and design theory.) Whatever, it’s a lovely piece of design.

2) Most of the knowledge bound up in the example is what design theorists Harold Nelson and Erik Stolterman call ‘knowledge of the real’ (rather than ‘knowledge of the (universally) true’). Of course, there is also ‘knowledge of the ideal’ – in the sense that Danica Savonick knows why this exercise is worth doing. But the design is replete with particulars– real things to get right – and has little truck with the illusory universal truths of learning theory. (“Group work is better than individual reflection”, “All classes should be flipped”, “Direct instruction beats discovery learning” etc.)

Nelson & Stolterman claim that design is the ‘first tradition’ in human development – before science and creative arts – and that it involves subtle inter-weavings of what is true, what is real and what is ideal. Skilled practice often involves design: we need to get better at recognising it and learning from it. Head for the hastac website now!

 

ACAD stands for ‘Activity-Centred Analysis and Design’

 

References/further reading

 

Nelson, H., & Stolterman, E. (2014). The design way: intentional change in an unpredictable world(2nd ed.). Cambridge MA: MIT Press.

Carvalho, L., & Goodyear, P. (Eds.). (2014). The architecture of productive learning networks. New York: Routledge.

Carvalho, L., Goodyear, P., & de Laat, M. (Eds.). (2017). Place-based spaces for networked learning. New York: Routledge.

Carvalho, L., & Yeoman, P. (2018). Framing learning entanglement in innovative learning spaces: Connecting theory, design and practice. British Educational Research Journal, 0(0). doi:doi:10.1002/berj.3483