Higher Education: an Owner's Guide – slides from my Times Higher Ed Student Festival keynote

Higher Education: an Owner’s Guide, slides from my keynote at the THE Student Festival, UK, 2021

With a possible audience of thousands, and without a clue they were there because the Zoom output was streamed to a different site (weird), I talked very fast about my experience of higher education for about 20 minutes at the THE UK Student Festival yesterday. The talk was recorded and will be used again for the THE Canada Student Festival later in the month. It’s a huge event – over 8000 enrolled (though not all attending every session) – with quite a lot of other keynotes and a great many other talks, panels, and discussions, aimed at helping students starting out in higher education. The audience was very different from those I normally talk to, and the  (very sensible) strict 20-minute format gave me a lot less leeway than the usual hour allowed, so I found it interestingly challenging. These are the slides I used.

The brief I was given was not to preach, but to share my experience of higher education, as a student and as an educator. My personal agenda was to talk about why and how online learning is worth doing (especially at AU), so I tried (a bit clunkily) to aim my story in that direction. I failed to remember to mention some key things, I spent too long on others, and I suspect that the most memorable message that came through was, for in-person students living on campus, to get a kettle (it’s a great way to make lots of new friends fast) but, hopefully, my bigger message got through to some. kettle

The essential point of my wild ramble was not that kettles are the solution to success in higher education, but that students and the rest of us should take ownership of our own education: it should be done by us, not to us. We should learn the way we want to learn, and we should learn what we want to learn; we should seek adventure and challenge rather than easy pickings; and we should hang out with interesting people and/or those we care about, because that’s how most learning happens, as well as being a large part of what makes it meaningful. I noted that we should focus on learning and should to try to ignore grades as much as possible, because grades destroy the love of doing something simply because we enjoy it. Essentially, my advice was about finding the things that intrinsically motivate us and reducing the effects of things that demotivate us. My own educational journey, and (I think) that of most committed educators, has largely followed that path: that’s how we thrived in a system not conducive to intrinsic motivation, and it’s the path we try to encourage our own students to take. I observed that it is much easier to own your own education when it is done online, at least if it is done in ways that take advantage of the medium, and not through a pale simulacrum of in-person teaching, because the teacher cannot be in control, the level of challenge is much more controllable, and there are way more people (online and in your own environment) who can support you.

Challenges of the Physical: slides from my keynote at XII Conferência Internacional de Tecnologias de Informação e Comunicação na Educação, September 2021

Here are the slides from my opening keynote today for the XII Conferência Internacional de Tecnologias de Informação e Comunicação na Educação in Portugal. first slide of the presentation

The conference theme was ‘challenges of the digital’ so I thought it might be fun to reverse the problem, and to think instead about the challenges of in-person education. In this presentation I imagined a world in which in-person teaching had never been invented, and presented a case for doing so. In fairness, it was not a very good case! But I did have fun using some of the more exotic voice changing features of my Voicelive Play vocal processor (which I normally use for performing music), presenting some of the arguments against my suggestions in different voices using a much better mic than my usual (pretty good) Blue Yeti. I might not use the special effects again that often, but I was quite impressed with the difference the better microphone made.

My central points (mostly implicit until the end) were:

  • That the biggest challenge of the digital is all the baggage that we have inherited from in-person teaching, and our continuing need to interoperate with in-person institutions.
  • That pedagogies are neither universal nor neutral. They are solutions to problems of learning in a particular context, in assembly with countless constraints and possibilities provided by that context: people, tools, structures, methods, systems, and so on.
  • That solutions to learning in a physical context – at least in the one-to-many model of traditional education systems – inevitably lead to a very strong power imbalance between teacher and learner, where the teacher is in control of every moment that the teaching event occurs. This has many repercussions, not least of which being that needs for autonomy and competence support are very poorly addressed (though relatedness comes for free), so it is really bad for intrinsic motivation.
  • Thus, the pedagogies of physical spaces have to compensate for the loss of control and achievable challenge that they naturally entail.
  • That the most common approach – and, again, an almost inevitable (i.e. the shortest path) follow-on from teaching a lot of people at once – involves rewards and punishments, that massively impair or destroy intrinsic motivation to learn and, in most cases, actively militate against effective learning.
  • That the affordances of teaching everyone the same thing at once lead fairly naturally to credentials for having learned it, often achieved in ‘efficient’ ways like proctored exams that are incredibly bad for learning, and that greatly reinforce the extrinsic motivation that is already highly problematic in the in-person modality. The credentials, not the learning, become the primary focus.
  • That support for autonomy and competence are naturally high in online learning, though support for relatedness is a mix of good and bad. There is no need for teachers being in control and, lacking most of the means of control available to in-person teachers, the only reliable way to regain it is through rewards and punishments which, as previously mentioned, are fatal to intrinsic motivation.
  • That the almost ubiquitous ways that distance educators inherit and use the pedagogies, methods, and structures of in-person learning – especially in the use of coercion through rewards and punishments (grades, credentials, etc) but also in schedules, fixed-length courses, inflexible learning outcomes, etc – are almost exactly the opposite of what its technologies can best support.

Towards the end, acknowledging that it is difficult to change such complex and deeply entangled systems (much though it is to be desired) I presented some ways of reducing the challenges of the physical in online teaching, and regaining that lost intrinsic motivation, that I summarized thus:

  • Let go (you cannot and should not control learning unless asked to do so), but stay close;
  • Make learning (not just its products) visible (and, in the process, better understand your teaching);
  • Make learning shared (cooperation and, where possible, collaboration built in from the ground up);
  • Don’t ever coerce (especially not through grades);
  • Care (for learners, for learning, for the subject).

It’s a theme that I have spoken and written of many, many times, but (apart from the last few slides) the way I presented it this time was new for me. I had fun pretending to be different people, and the audience seemed to like it, in a challenging kind of a way. There were some great questions at the end, not all of which I had time to answer, though I’m happy to continue the conversation here, or via Twitter.

Why do we work from home but learn remotely?

I am slowly getting used to the ugly abbreviation WFH that has emerged during the pandemic, though I don’t much like it because it’s not always accurate. Even in pandemic times I often work from my boat (WFB). In non-pandemic times I’ve worked from a tent (WFT), a library (WFL), a hotel room (WFHR), a park bench (WFPB), a conference (WFC), a plane (WFP), a bus (WF… OK, you get the picture), and much, much more. I have even worked at Athabasca University’s own buildings (Working from Work?) on rare occasions. But why do most of us in the trade so rarely use terms like learning from home when working from home (WFH) is so ubiquitous?

Terms like e-learning, online learning, distance learning, remote learning, and so on, are weird. Learning is never remote, electronic, online, or at a distance.  There is more sense to terms like distance education, online education, remote teaching, and so on, because education and teaching describe relationships between people, and there are different ways that those relationships can be mediated, that do (or should) deeply affect the process. There is also a whole slew of intentional and implicit structures, systems, methods, and toolsets that are assumed when we prefix education with terms like distance or online. But why online or distance learning?

As teachers we are (rightly) taught that it’s not about the teaching, it’s about the learning. For at least the last 30 years or more we have, for instance, therefore been strongly encouraged to use the term ‘learning & teaching’ instead of ‘teaching & learning’ because learning must come first. I’ve corrected people myself for getting the order wrong, many times. Charitably, therefore, it might be that we are trying to draw attention to the fact that it’s about learning. But, if so, why distance or online?

Ricardo Liberato, CC BY-SA 2.0 via Wikimedia Commons I think something nasty has happened to the term ‘learning’ when it is used this way, because I think that what we actually mean by it is ‘teaching’.  Some British English dialects take that dubious elision fully on board. When something nasty happens to someone as a consequence of something they have done that is perceived to be wrong, or even when some punishment is inflicted on them by someone else, it is common in some circles to say ‘that’ll learn yer’ (the ‘yer’ is important – don’t imagine the Queen saying in received pronunciation ‘that will learn you’ because it would be wrong). When I hear the phrase I imagine it being said with a snarl. It’s a cruel thing to say, though it can be used kind-of humorously, at least if, as many of my compatriots do, you appreciate a particularly crude form of Benny-Hillish shadenfreude (‘Ha ha, you fell flat on your face and hurt yourself. That’ll learn yer’).

Outside a subset of British and perhaps some other minor English vernaculars, learning is never something that we do to people. It’s something done by people, with what and with whom is around them (and that might include a teaching website, textbook, or course pack). So let’s stop calling people distance or online learners because it devalues and obscures what they are actually doing. They are not being learned at. They are being taught at a distance, and learning from home (or wherever they happen to be).

Mediaeval Teaching in the Digital Age (slides from my keynote at Oxford Brookes University, May 26, 2021)

 front slide, mediaeval teaching

These are the slides from my keynote today at the Oxford Brookes “Theorizing the Virtual” School of Education Research Conference. As theorizing the virtual is pretty much my thing, I was keen to be a part of this! It was an ungodly hour of the day for me (2am kickoff) but it was worth staying up for. It was a great bunch of attendees who really got into the spirit of the thing and kept me wide awake. I wish I could hang around for the rest of it but, on the bright side, at least I’m up at the right time to see the Super Flower Blood Moon (though it’s looking cloudy, darn it).  In this talk I dwelt on a few of the notable differences between online and in-person teaching. This is the abstract…

Pedagogical methods (ways of teaching) are solutions to problems of helping people to learn, in a context filled with economic, physical, temporal, legal, moral, social, political, technological, and organizational constraints. In mediaeval times books were rare and unaffordable, and experts’ time was precious and limited, so lectures were a pragmatic solution, but they in turn created more problems. Counter-technologies such as classes, classrooms, behavioural rules and norms, courses, terms, curricula, timetables and assignment deadlines were were devised to solve those problems, then methods of teaching (pedagogies) were in turn invented to solve problems these counter-technologies caused, notably including:
· people who might not want (or be able) to be there at that time,
· people who were bored and
· people who were confused.
Better pedagogies supported learner needs for autonomy and competence, or helped learners find relevance to their own goals, values, and interests. They exploited physical closeness for support, role-modelling, inspiration, belongingness and so on. However, increasingly many relied on extrinsic motivators, like classroom discipline, grades and credentials to coerce students to learn. Extrinsic motivation achieves compliance, but it makes the reward or avoidance of the punishment the goal, persistently and often permanently crowding out intrinsic motivation. Intelligent students respond with instrumental approaches, satisficing, or cheating. Learning seldom persists; love of the subject is subdued; learners learn to learn in ineffective ways. More layers of counter-technologies are needed to limit the damage, and so it goes on.
Online, the constraints are very different, and its native forms are the motivational inverse of in-person learning. An online teacher cannot control every moment of a learner’s time, and learners can use the freedoms they gain to take the time they need, when they need it, to learn and to reflect, without the constraints of scheduled classroom hours and deadlines. However, more effort is usually needed to support their needs for relatedness. Unfortunately, many online teachers try (or are required) to re-establish the control they had in the classroom through grading or the promise of credentials, recreating the mediaeval problems that would otherwise not exist, using tools like learning management systems that were designed (poorly) to replicate in-person teaching functions. These are solutions to the problems caused by counter-technologies, not to problems of learning.
There are better ways, and that’s what this session is about.

front slide, mediaeval teaching

Educational technology: what it is and how it works

https://rdcu.be/ch1tl

This is a link to my latest paper in the journal AI & Society. You can read it in a web browser from there, but it is not directly downloadable. A preprint of the submitted version (some small differences and uncorrected errors here and there, notably in citations) can be downloaded from https://auspace.athabascau.ca/handle/2149/3653. The published version should be downloadable for free by Researchgate members.

This is a long paper (about 10,000 words), that summarizes some of the central elements of the theoretical model of learning, teaching and technology developed in my recently submitted book (still awaiting review) and that gives a few examples of its application. For instance, it explains:

  • why, on average researchers find no significant difference between learning with and without tech.
  • why learning styles theories are a) inherently unprovable, b) not important even if they were, and c) a really bad idea in any case.
  • why bad teaching sometimes works (and, conversely, why good teaching sometimes fails)
  • why replication studies cannot be done for most educational interventions (and, for the small subset that are susceptible to reductive study, all you can prove is that your technology works as intended, not whether it does anything useful).

Abstract

This theoretical paper elucidates the nature of educational technology and, in the process, sheds light on a number of phenomena in educational systems, from the no-significant-difference phenomenon to the singular lack of replication in studies of educational technologies.  Its central thesis is that we are not just users of technologies but coparticipants in them. Our participant roles may range from pressing power switches to designing digital learning systems to performing calculations in our heads. Some technologies may demand our participation only in order to enact fixed, predesigned orchestrations correctly. Other technologies leave gaps that we can or must fill with novel orchestrations, that we may perform more or less well. Most are a mix of the two, and the mix varies according to context, participant, and use. This participative orchestration is highly distributed: in educational systems, coparticipants include the learner, the teacher, and many others, from textbook authors to LMS programmers, as well as the tools and methods they use and create.  From this perspective,  all learners and teachers are educational technologists. The technologies of education are seen to be deeply, fundamentally, and irreducibly human, complex, situated and social in their constitution, their form, and their purpose, and as ungeneralizable in their effects as the choice of paintbrush is to the production of great art.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/8692242/my-latest-paper-educational-technology-what-it-is-and-how-it-works

Are experienced online teachers best-placed to help in-person teachers cope with suddenly having to teach online? Maybe not.

lecturingI recently downloaded What Teacher Educators Should Have Learned From 2020. This is an open edited book, freely downloadable from the AACE site, for teachers of teachers whose lives were disrupted by the sudden move to emergency remote teaching over the past year or so.  I’ve only skimmed the contents and read a couple of the chapters, but my first impressions are positive. Edited by Richard Ferdig and Kristine Pytash, It springs from the very active and engaged AACE SITE community, which is a good indicator of expertise and experience. It seems well organized into three main sections:

  1.         Social and Emotional Learning for Teacher Education.
  2.         Online Teaching and Learning for Teacher Education.
  3.         eXtended Reality (XR) for Teacher Education

I like the up-front emphasis on social and emotional aspects, addressing things like belongingness, compassion, and community, mainly from theoretical/model-oriented perspectives, and the other sections seem wisely chosen to meet practitioner needs. The chapters adopt a standardized structure:

  • Introduction. 
  • What We Know. 
  • Lessons Learned for Research. 
  • Lessons Learned for Practice. 
  • What You Should Read. 
  • References

Again, this seems pretty sensible, maintaining a good focus on actionable knowledge and practical steps to be taken. It’s not quite a textbook, but it’s a useful teach-yourself resource with good coverage. I look forward to dipping into it a bit more deeply. I expect to find some good ideas, good practices, and good theoretical models to support my teaching and my understanding of the issues. And I’m really pleased that it is being released as an open publication: well done, AACE, for making this openly available.

But I do wonder a little about who else will read this.

Comfort zones and uncomfortable zones

The other day I was chatting with a neighbour who teaches a traditional hard science subject at one of the local universities, who was venting about the problems of teaching via Zoom. He knew that I had a bit of interest and experience in this area, so he asked whether I had any advice. I started to suggest some ways of rethinking it as a pedagogical opportunity, but he was not impressed. Even something as low-threshold and straightforward as flipping the classroom or focusing on what students do rather than what he has to tell them was a step too far. He patiently explained that he has classes with hundreds of students and fixed topics that they need to learn, and he really didn’t see it as desirable or even possible to depart from his well-tried lecture format. At least it would be too much work and he didn’t have the time for it. I did try to push back on that a bit and I may have mentioned the overwhelming body of research that suggests this might not be a wise move, but he was pretty clear and firm about this.  What he actually wanted was for someone to make (or tell him how to make) the digital technology as easy and as comfortably familiar as the lecture theatre, and that would somehow make the students as engaged as he perceived them to normally be in his lectures, without notably changing how he taught. The problem was the darn technology, not the teaching. I bit my tongue at this point. I eventually came up with a platitude or two about trying to find different ways to make learning visible, about explicitly showing that he cares, about taking time to listen, about modelling the behaviour he wanted to see, about using the chat to good advantage, and about how motivation differs online and off, but I don’t think it helped. I suspect that the only things that really resonated with him were suggestions about how to get the most out of a webcam and a recommendation to get a better microphone.

Within the context in which he usually teaches, he is probably a very good teacher. He’s a likeable person who clearly cares a lot about his students, he knows a lot about his subject, and he knows how to make it appealing within the situation that he normally works. His courses, as he described them, are very conventional, relying a lot on the structure given to them by the industry-driven curriculum and the university’s processes, norms, and structures, and he fills his role in all that admirably. I think he is pretty typical of the vast majority of teachers. They’re good at what they do, comfortable with how they do it, and they just want the technology to accommodate them continuing to do so without unnecessary obstacles.

Unfortunately, technology doesn’t work that way.

The main reason it doesn’t work is very simple: technologies (including pedagogies) affect one another in complex and recursive ways, so (with some trivial exceptions) you can’t change one element (especially a large element) and expect the rest to work as they did before.  It’s simple, intuitive, and obvious but unless you are already well immersed in both systems theories and educational theory, really taking it to heart and understanding how it must affect your practice demands a pretty big shift in weltanschauung, which is not the kind of thing I was keen to start while on my way to the store in the midst of a busy day.

To make matters worse, even if teachers do acknowledge the need to change, their assumption that things will eventually (maybe soon) return to normal means that they are – reasonably enough –  not willing and probably not able to invest a lot of time into it. A big part of the reason for this is that, thanks to the aforementioned interdependencies, they are probably running round like blue-arsed flies just trying to keep things together, and filling their time with fixing the things that inevitably break in the process. Systems thrive on this kind of self-healing feedback loop. I guess teachers figure that, if they can work out how to tread water until the pandemic has run its course, it will be OK in the end.

If only.

Why in-person education works

The hallmark technologies (mandatory lectures, assignments, grades, exams, etc, etc) of in-person teaching are worse than awful but, just as a talented musician can make beautiful noises with limited technical knowledge and sub-standard instruments, so there are countless teachers who use atrocious methods in dreadful contexts but who successfully lead their students to learn. As long as the technologies are soft and flexible enough to allow them to paper over the cracks of bad tools and methods with good technique, talent, and passion, it works well enough for enough people enough of the time and can (with enough talent and passion) even be inspiring.

It would not work at all, though, without the massive machinery that surrounds it.

An institution (including its systems, structures, and tools) is itself designed to teach, no matter how bad the teachers are within it. The opportunities for students to learn from and with others around them, including other students, professors, support staff, administrators, and so on; the supporting technologies, including rules, physical spaces, structures, furnishings, and tools; the common rooms, the hallways, the smokers’ areas (best classrooms ever), the lecture theatres, the bars and the coffee shops; the timetables that make students physically travel to a location together (and thus massively increase salience); the notices on the walls; the clubs and societies; the librarians, the libraries, the students reading and writing within those libraries, echoing and amplifying the culture of learning that pervades them; the student dorms and shared kitchens where even more learning happens; the parties; even the awful extrinsic motivation of grades, teacher power, and norms and rules of behaviour that emerged in the first place due to the profound motivational shortcomings of in-person teaching. All of this and more conspires to support a basic level of at least mediocre (but good enough) learning, whether or not teachers teach well. It’s a massively distributed technology enacted by many coparticipants, of which designated teachers are just a part, and in which students are the lead actors among a cast of thousands. Online, those thousands are often largely invisible. At best, their presence tends to be highly filtered, channeled, or muted.

Why in-person methods don’t transfer well online

When most of that massive complex machinery is suddenly removed, leaving nothing but a generic interface better suited to remote business meetings than learning or, much worse, some awful approximation of all the evil, hard, disempowering technologies of traditional teaching wrapped around Zoom, or nightmarishly inhuman online proctoring systems, much of the teaching (in the broadest sense) disappears with it. Teaching in an institution is not just what teachers do. It’s the work of a community; of all the structures the community creates and uses; of the written and unwritten rules; of the tacit knowledge imparted by engagement in a space made for learning; of the massive preparation of schooling and the intricate loops that connect it with the rest of society; of attitudes and cultures that are shaped and reinforced by all the rest.  It’s no wonder that teachers attempting to transfer small (but the most visible) parts of that technology online struggle with it. They need to fill the ever-widening gaps left when most of the comfortable support structures of in-person institutions that made it possible in the first place are either gone or mutated into something lean and hungry. It can be done, but it is really hard work.

More abstractly, a big part of the problem with this transfer-what-used-to-work-in-person approach is that it is a technology-first approach to the problem that focuses on one technology rather than the whole. The technology of choice in this case happens to be a set of pedagogical methods, but it is no different in principle than picking a digital tool and letting that decide how you will teach. Neither makes much sense. All the technologies in the assembly – including pedagogies, digital tools, regulations, designs, and structures – have to work together. No single technology has precedence, beyond the one that results from assembling the rest. To make matters worse, what-used-to-work-in-person pedagogies were situated solutions to the problems of teaching in physical classrooms, not universally applicable methods of teaching. Though there are some similarities here and there, the problems of teaching online are not at all the same as those of in-person teaching so of course the solutions are different. Simply transferring in-person pedagogies to an online context is much like using the paddles from a kayak to power a bicycle. You might move, but you won’t move far, you won’t move fast, you won’t move where you want to go, and it is quite likely to end in injury to yourself or others.

Such problems have, to a large extent, been adequately solved by teachers and institutions that work primarily online. Online institutions and organizations have infrastructure, processes, rules, tools, cultures, and norms that have evolved to work together, starting with the baseline assumption that little or none of the physical stuff will ever be available. Anything that didn’t work never made it to first base, or has not survived. Those that have been around a while might not be perfect, but they have ironed out most of the kinks and filled in most of the gaps. Most of my work, and that of my smarter peers, begins in this different context. In fact, in my case, it mainly involves savagely critiquing that context and figuring out ways to improve it, so it is yet another step removed from where in-person teachers are now.

OK, maybe I could offer a little advice or, at least, a metaphor

Roughly 20 years ago I did share a similar context. Working in an in-person university, I had to lead a team of novice online teachers from geographically dispersed colleges to create and teach a blended program with 28 new online courses. We built the whole thing in 6 months from start to finish, including the formal evaluations and approvals process. I could share some generic lessons from what I discovered then, the main one being to put most of the effort into learning to teach online, not into designing course materials. Put dialogue and community first, not structure. For instance, make the first thing students see in the LMS the discussion, not your notes or slides, and use the discussion to share content and guide the process. However, I’d mostly feel like the driver of a Model T Ford trying to teach someone to drive a Tesla. Technologies have changed, I have changed, my memory is unreliable.

bicycleIn fact, I haven’t driven a car of any description in years. What I normally do now is, metaphorically, much closer to riding a bicycle, which I happen to do and enjoy a lot in real life too. A bike is a really smart, well-adapted, appropriate, versatile, maintainable, sustainable soft technology for getting around. The journey tends to be much more healthy and enjoyable, traffic jams don’t bother you, you can go all sorts of places cars cannot reach, and you can much more easily stop wherever you like along the way to explore what interests you. You can pretty much guarantee that you will arrive when and where you planned to arrive, give or take a few minutes. In the city, it’s often the fastest way to get around, once you factor in parking etc. It’s very liberating. It is true that more effort is needed to get from A to B, bad weather can be a pain, and it would not be the fastest or most comfortable way to reach the other side of the continent: sometimes, alternative forms of transport are definitely worth taking and I’m not against them when it’s appropriate to use them. And the bike I normally ride does have a little electric motor in one of the wheels that helps push me up hills (not much, but enough) but it doesn’t interfere with the joy (or most of the effort) of riding.  I have learned that low-threshold, adaptable, resilient systems are often much smarter in many ways than high-tech platforms because they are part-human. They can take on your own smartness and creativity in ways no amount of automation can match. This is true of online learning tools as much as it is true of bicycles. Blogs, wikis, email, discussion forums, and so on often beat the pants off learning management systems, commercial teaching platforms, learning analytics tools or AI chatbots for many advanced pedagogical methods because they can become what you want them to be, rather than what the designer thought you wanted, and they can go anywhere, without constraint. Of course, the flip side is that they take more effort, sometimes take more time, and (without enormous care) can make it harder for all concerned to do things that are automated and streamlined in more highly engineered tools, so they might not always be the best option in all circumstances, any more than a bike is the best way to get up a snowy mountain or to cross an ocean.

Why you shouldn’t listen to my advice

It’s sad but true that most of what I would really like to say on the subject of online learning won’t help teachers on the ground right now, and it is actually worse than the help their peers could give them because what I really want to tell them is to change everything and to see the world completely differently. That’s pretty threatening, especially in these already vulnerable times, and not much use if you have a class to teach tomorrow morning.

The AACE book is more grounded in where in-person teachers are now. The chapter “We Need to Help Teachers Withstand Public Criticism as They Learn to Teach Online”, for example, delves into the issues well, in accessible ways that derive from a clear understanding of the context.  However, the book cannot help but be an implicit (and, often, explicit) critique of how teachers currently teach: that’s implied in the title, and in the chapter structures.  If you’re already interested enough in the subject and willing enough to change how you teach that you are reading this book in the first place, then this is great. You are 90% of the way there already, and you are ready to learn those lessons. One of the positive sides of emergency remote teaching has been that it has encouraged some teachers to reflect on their teaching practices and purposes, in ways that will probably continue to be beneficial if and when they return to in-person teaching. They will enjoy this book, and they may be the intended audience. But they are not the ones that really need it.

I would quite like to see (though maybe not to read) a different kind of book containing advice from beginners. Maybe it would have a title something like ‘What I learned in 2020’ or ‘How I survived Zoom.’ Emergency remote teachers might be more inclined to listen to the people who didn’t know the ‘right’ ways of doing things when the crisis began, who really didn’t want to change, who maybe resented the imposition, but who found ways to work through it from where they were then, rather than where the experts think (or know) they should be aiming now. It would no doubt annoy me and other distance learning researchers because, from the perspective of recognized good practice, much of it would probably be terrible but, unlike what we have to offer, it would actually be useful. A few chapters in the AACE book are grounded in concrete experience of this nature, but even they wind up saying what should have happened, framing the solutions in the existing discourse of the distance learning discipline. Most chapters consist of advice from experts who already knew the answers before the pandemic started. It is telling that the word ‘should’ occurs a lot more frequently than it should. This is not a criticism of the authors or editors of the book: the book is clear from the start that it is going to be a critique of current practice and a practical guidebook to the territory, and most of the advice I’ve seen in it so far makes a lot of sense. It’s just not likely to affect many of the ones who have no wish to change not just their practices but their fundamental attitudes to teaching. Sadly, that’s also true of this post which, I think, is therefore more of an explanation of why I’ve been staring into the headlights for most of the pandemic, rather than a serious attempt to help those in need. I hope there’s some value in that because it feels weird to be a (slight, minor, still-learning) expert in the field with very strong opinions about how online learning should work, but to have nothing useful to say on the subject at the one time it ought to have the most impact.

Read the book:

Ferdig, R.E. & Pytash, K.E. (2021). What Teacher Educators Should Have Learned From 2020. Association for the Advancement of Computing in Education (AACE). Retrieved March 22, 2021 from https://www.learntechlib.org/primary/p/219088/.

EdTech Books

This is a great, well presented and nicely curated selection of open books on education and educational technology, ranging from classics (and compilations of chapters by classic authors) to modern guides, textbooks, and blog compilations, covering everything from learning theory to choice of LMS. Some are peer-reviewed, there’s a mix of licences from PD to restrictive CC , and there’s good guidance provided about the type and quality of content. There’s also support for collaboration and publication. All books are readable online, most can be downloaded as (at least) PDF. I think the main target audience is students of education/online learning, and practitioners – at least, there’s a strong practical focus.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/7161867/edtech-books (where you can find some really interesting comments, including the one that my automated syndicator mistakenly turned into the main post the first time it ran)

Joyful assessment: beyond high-stakes testing

Here are my slides from my presentation at the Innovate Learning Summit yesterday. It’s not world-shattering stuff – just a brutal attack on proctored, unseen written exams (PUWEs, pronounced ‘pooies’), followed by a description of the rationale, process, benefits, and unwanted consequences behind the particular portfolio-based approach to assessment employed in most of my teaching. It includes a set of constraints that I think are important to consider in any assessment process, grouped into pedagogical, motivational, and housekeeping (mainly relating to credentials) clusters. I list 13 benefits of my approach relating to each of those clusters, which I think make a pretty resounding case for using it instead of traditional assignments and tests. However, I also discuss outstanding issues, most of which relate to the external context and expectations of students or the institution, but a couple of which are fairly fundamental flaws (notably the extreme importance of prompt, caring, helpful instructor/tutor engagement in making it all work, which can be highly problematic when it doesn’t happen) that I am still struggling with.

Skills lost due to COVID-19 school closures will hit economic output for generations (hmmm)

Snippet from OECD report on covid-19 and education This CBC report is one of many dozens of articles in the world’s press highlighting one rather small but startling assertion in a recent OECD report on the effects of Covid-19 on education – that the ‘lost’ third of a year of schooling in many countries will lead to an overall lasting drop in GDP of 1.5% across the world. Though it contains many more fascinating and useful insights that are far more significant and helpful, the report itself does make this assertion quite early on and repeats it for good measure, so it is not surprising that journalists have jumped on it. It is important to observe, though, that the reasoning behind it is based on a model developed by Hanushek and Woessman over several years, and an unpublished article by the authors that tries to explain variations in global productivity according to amount and  – far more importantly – the quality of education: that long-run productivity is a direct consequence of the cognitive skills (or knowledge capital) of a nation, that can be mapped directly to how well and how much the population is educated.

As an educator I find this model, at a glance, to be reassuring and confirmatory because it suggests that we do actually have a positive effect on our students. However, there may be a few grounds on which it might be challenged (disclaimer: this is speculation). The first and most obvious is that correlation does not equal causation. The fact that countries that do invest in improving education consistently see productivity gains to match in years to come is interesting, but it raises the question of what led to that investment in the first place and whether that might be the ultimate cause, not the education itself.  A country that has invested in increasing the quality of education would, normally, be doing so as a result of values and circumstances that may lead to other consequences and/or be enabled by other things (such as rising prosperity, competition from elsewhere, a shift to more liberal values, and so on).  The second objection might be that, sure, increased quality of education does lead to greater productivity, but that it is not the educational process that is causing it, as such. Perhaps, for instance, an increased focus on attainment raises aspirations. A further objection might be that the definition of ‘quality’ does not measure what they think it measures. A brief skim of the model used suggests that it makes extensive use of scores from the likes of TIMSS, PIRLS and PISA, standardized test approaches used to compare educational ‘effectiveness’ in different regions that embody quite a lot of biases, are often manipulated at a governmental level, and that, as I have mentioned once or twice before, are extremely dubious indicators of learning: in fact, even when they are not manipulated, they may indicate willingness to comply with the demands of the powerful more than learning (does that improve GDP? Probably).  Another objection might be that absence of time spent in school does not equate to absence of education. Indeed, Hanushek and Woessman’s central thesis is that it is not the amount but the quality of schooling that matters, so it seems bizarre that they might fall back on quantifying learning by time spent in school. We know for sure that, though students may not have been conforming to curricula at the rate desired by schools and colleges, they have not stopped learning. In fact, in many ways and in many places, there are grounds to believe that there have been positive learning benefits: better family learning, more autonomy, more thoughtful pedagogies, more intentional learning community forming, and so on.  Out of this may spring a renewed focus on how people learn and how best to support them, rather than maintaining a system that evolved in mediaeval times to support very different learning needs, and that is so solidly packed with counter technologies and so embedded in so many other systems that have nothing to do with learning that we have lost sight of the ones that actually matter. If education improves as a result, then (if it is true that better and more education improves the bottom line) we may even see gains in GDP. I expect that there are other reasons for doubt: I have only skimmed the surface of the possible concerns.

I may be wrong to be sceptical –  in fairness, I have not read the many papers and books produced by Hanushek and Woessman on the subject, I am not an economist, nor do I have sufficient expertise (or interest) to analyze the regression model that they use. Perhaps they have fully addressed such concerns in that unpublished paper and the simplistic cause-effect prediction distorts their claims. But, knowing a little about complex adaptive systems, my main objection is that this is an entirely new context to which models that have worked before may no longer apply and that, even if they do, there are countless other factors that will affect the outcome in both positive and negative ways, so this is not so much a prediction as an observation about one small part of a small part of a much bigger emergent change that is quite unpredictable. I am extremely cautious at the best of times whenever I see people attempting to find simple causal linear relationships of this nature, especially when they are so precisely quantified, especially when past indicators are applied to something wholly novel that we have never seen before with such widespread effects, especially given the complex relationships at every level, from individual to national.  I’m glad they are telling the story – it is an interesting one that no doubt contains grains of important truths – but it is just an informative story, not predictive science.  The OECD has a bit of track record on this kind of misinterpretation, especially in education. This is the same organization that (laughably, if it weren’t so influential) claimed that educational technology in the classroom is bad for learning. There’s not a problem with the data collection or analysis, as such. The problem is with the predictions and recommendations drawn from it.

Beyond methodological worries, though, and even if their predictions about GDP are correct (I am pretty sure they are not – there are too many other factors at play, including huge ones like the destruction of the environment that makes the odd 1.5% seem like a drop in the barrel) then it might be a good thing. It might be that we are moving – rather reluctantly – into a world in which GDP serves as an even less effective measure of success than it already is. There are already plentiful reasons to find it wanting, from its poor consideration of ecological consequences to its wilful blindness to (and causal effect upon) inequalities, to its simple inadequacy to capture the complexity and richness of human culture and wealth. I am a huge fan of the state of Bhutan’s rejection of the GDP, that it has replaced with the GNH happiness index. The GNH makes far more sense, and is what has led Bhutan to be one of the only countries in the world to be carbon positive, as well as being (arguably but provably) one of the happiest countries in the world. What would you rather have, money (at least for a few, probably not you), or happiness and a sustainable future? For Bhutan, education is not for economic prosperity: it is about improving happiness, which includes good governance, sustainability, and preservation of (but not ossification of) culture.

Many educators – and I am very definitely one of them – share Bhutan’s perspective on education. I think that my customer is not the student, or a government, or companies, but society as a whole, and that education makes (or should make) for happier, safer, more inventive, more tolerant, more stable, more adaptive societies, as well as many other good things. It supports dynamic meta-stability and thus the evolution of culture. It is very easy to lose sight of that goal when we have to account to companies, governments, other institutions, and to so many more deeply entangled sets of people with very different agendas and values, not to mention our inevitable focus on the hard methods and tools of whatever it is that we are teaching, as well as the norms and regulations of wherever we teach it. But we should not ever forget why we are here. It is to make the world a better place, not just for our students but for everyone. Why else would we bother?

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6578662/skills-lost-due-to-covid-19-school-closures-will-hit-economic-output-for-generations-hmmm

Evaluating assessment

Exam A group of us at AU have begun discussions about how we might transform our assessment practices, in the light of the far-reaching AU Imagine plan and principles. This is a rare and exciting opportunity to bring about radical and positive change in how learning happens at the institution. Hard technologies influence soft more than vice versa, and assessments (particularly when tied to credentials) tend to be among the hardest of all technologies in any pedagogical intervention. They are therefore a powerful lever for change. Equally, and for the same reasons, they are too often the large, slow, structural elements that infest systems to stunt progress and innovation.

Almost all learning must involve assessment, whether it be of one’s own learning, or provided by other people or machines. Even babies constantly assess their own learning. Reflection is assessment. It is completely natural and it only gets weird when we treat it as a summative judgment, especially when we add grades or credentials to the process, thus normally changing the purpose of learning from achieving competence to achieving a reward. At best it distorts learning, making it seem like a chore rather than a delight, at worst it destroys it, even (and perhaps especially) when learners successfully comply with the demands of assessors and get a good grade. Unfortunately, that’s how most educational systems are structured, so the big challenge to all teachers must be to eliminate or at least to massively reduce this deeply pernicious effect. A large number of the pedagogies that we most value are designed to solve problems that are directly caused by credentials. These pedagogies include assessment practices themselves.

With that in mind, before the group’s first meeting I compiled a list of some of the main principles that I adhere to when designing assessments, most of which are designed to reduce or eliminate the structural failings of educational systems. The meeting caused me to reflect a bit more. This is the result:

Principles applying to all assessments

  • The primary purpose of assessment is to help the learner to improve their learning. All assessment should be formative.
  • Assessment without feedback (teacher, peer, machine, self) is judgement, not assessment, pointless.
  • Ideally, feedback should be direct and immediate or, at least, as prompt as possible.
  • Feedback should only ever relate to what has been done, never the doer.
  • No criticism should ever be made without also at least outlining steps that might be taken to improve on it.
  • Grades (with some very rare minor exceptions where the grade is intrinsic to the activity, such as some gaming scenarios or, arguably, objective single-answer quizzes with T/F answers) are not feedback.
  • Assessment should never ever be used to reward or punish particular prior learning behaviours (e.g. use of exams to encourage revision, grades as goals, marks for participation, etc) .
  • Students should be able to choose how, when and on what they are assessed.
  • Where possible, students should participate in the assessment of themselves and others.
  • Assessment should help the teacher to understand the needs, interests, skills, and gaps in knowledge of their students, and should be used to help to improve teaching.
  • Assessment is a way to show learners that we care about their learning.

Specific principles for summative assessments

A secondary (and always secondary) purpose of assessment is to provide evidence for credentials. This is normally described as summative assessment, implying that it assesses a state of accomplishment when learning has ended. That is a completely ridiculous idea. Learning doesn’t end. Human learning is not in any meaningful way like programming a computer or storing stuff in a database. Knowledge and skills are active, ever-transforming, forever actively renewed, reframed, modified, and extended. They are things we do, not things we have.

With that in mind, here are my principles for assessment for credentials (none of which supersede or override any of the above core principles for assessment, which always apply):

  • There should be no assessment task that is not in itself a positive learning activity. Anything else is at best inefficient, at worst punitive/extrinsically rewarding.
  • Assessment for credentials must be fairly applied to all students.
  • Credentials should never be based on comparisons between students (norm-referenced assessment is always, unequivocally, and unredeemably wrong).
  • The criteria for achieving a credential should be clear to the learner and other interested parties (such as employers or other institutions), ideally before it happens, though this should not forestall the achievement and consideration of other valuable outcomes.
  • There is no such thing as failure, only unfinished learning. Credentials should only celebrate success, not punish current inability to succeed.
  • Students should be able to choose when they are ready to be assessed, and should be able to keep trying until they succeed.
  • Credentials should be based on evidence of competence and nothing else.
  • It should be impossible to compromise an assessment by revealing either the assessment or solutions to it.
  • There should be at least two ways to demonstrate competence, ideally more. Students should only have to prove it once (though may do so in many ways and many times, if they wish).
  • More than one person should be involved in judging competence (at least as an option, and/or on a regularly taken sample).
  • Students should have at least some say in how, when, and where they are assessed.
  • Where possible (accepting potential issues with professional accreditation, credit transfer, etc) they should have some say over the competencies that are assessed, in weighting and/or outcome.
  • Grades and marks should be avoided except where mandated elsewhere. Even then, all passes should be treated as an ‘A’ because students should be able to keep trying until they excel.
  • Great success may sometimes be worthy of an award – e.g. a distinction – but such an award should never be treated as a reward.
  • Assessment for credentials should demonstrate the ability to apply learning in an authentic context. There may be many such contexts.
  • Ideally, assessment for credentials should be decoupled from the main teaching process, because of risks of bias, the potential issues of teaching to the test (regardless of individual needs, interests and capabilities) and the dangers to motivation of the assessment crowding out the learning. However, these risks are much lower if all the above principles are taken on board.

I have most likely missed a few important issues, and there is a bit of redundancy in all this, but this is a work in progress. I think it covers the main points.

Further random reflections

There are some overriding principles and implied specifics in all of this. For instance, respect for diversity, accessibility, respect for individuals, and recognition of student control all fall out of or underpin these principles. It implies that we should recognize success, even when it is not the success we expected, so outcome harvesting makes far more sense than measurement of planned outcomes. It implies that failure should only ever be seen as unfinished learning, not as a summative judgment of terminal competence, so appreciative inquiry is far better than negative critique. It implies flexibility in all aspects of the activity. It implies, above and beyond any other purpose, that the focus should always be on learning. If assessment for credentials adversely affects learning then it should be changed at once.

In terms of implementation, while objective quizzes and their cousins can play a useful formative role in helping students to self-assess and to build confidence, machines (whether implemented by computers or rule-following humans) should normally be kept out of credentialling. There’s a place for AI but only when it augments and informs human intelligence, never when it behaves autonomously. Written exams and their ilk should be avoided, unless they conform to or do not conflict with all the above principles: I have found very few examples like this in the real world, though some practical demonstrations of competence in an authentic setting (e.g. lab work and reporting) and some reflective exercises on prior work can be effective.

A portfolio of evidence, including a reflective commentary, is usually going to be the backbone of any fair, humane, effective assessment: something that lets students highlight successes (whether planned or not), that helps them to consolidate what they have learned, and that is flexible enough to demonstrate competence shown in any number of ways. Outputs or observations of authentic activities are going to be important contributors to that. My personal preference in summative assessments is to only use the intended (including student-generated) and/or harvested outcomes for judging success, not for mandated assignments. This gives flexibility, it works for every subject, and it provides unquivocal and precise evidence of success. It’s also often good to talk with students, perhaps formally (e.g. a presentation or oral exam), in order to tease out what they really know and to give instant feedback. It is worth noting that, unlike written exams and their ilk, such methods are actually fun for all concerned, albeit that the pleasure comes from solving problems and overcoming challenges, so it is seldom easy.

Interestingly, there are occasions in traditional academia where these principles are, for the most part, already widely applied. A typical doctoral thesis/dissertation, for example, is often quite close to it (especially in more modern professional forms that put more emphasis on recording the process), as are some student projects. We know that such things are a really good idea, and lead to far richer, more persistent, more fulfilling learning for everyone. We do not do them ubiquitously for reasons of cost and time. It does take a long time to assess something like this well, and it can take more time during the rest of the teaching process thanks to the personalization (real personalization, not the teacher-imposed form popularized by learning analytics aficionados) and extra care that it implies. It is an efficient use of our time, though, because of its active contribution to learning, unlike a great many traditional assessment methods like teacher-set assignments (minimal contribution) and exams (negative contribution).  A lot of the reason for our reticence, though, is the typical university’s schedule and class timetabling, which makes everything pile on at once in an intolerable avalanche of submissions. If we really take autonomy and flexibility on board, it doesn’t have to be that way. If students submit work when it is ready to be submitted, if they are not all working in lock-step, and if it is a work of love rather than compliance, then assessment is often a positively pleasurable task and is naturally staggered. Yes, it probably costs a bit more time in the end (though there are plenty of ways to mitigate that, from peer groups to pedagogical design) but every part of it is dedicated to learning, and the results are much better for everyone.

Some useful further reading

This is a fairly random selection of sources that relate to the principles above in one way or another. I have definitely missed a lot. Sorry for any missing URLs or paywalled articles: you may be able to find downloadable online versions somewhere.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. Retrieved from https://www.jhsph.edu/departments/population-family-and-reproductive-health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-learning.pdf

Boud, D. (2007). Reframing assessment as if learning were important. Retrieved from https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in organizational change and development, 1, 129-169.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3/4), 325-346.

Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3(3), 220-233.

Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes (Kindle ed.). Mariner Books. (this one is worth forking out money for).

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. Retrieved from http://www.alfiekohn.org/blogs/personalized/ (check out Alfie Kohn’s whole site for plentiful other papers and articles – consistently excellent).

Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-Determination research (pp. 183-203). Rochester, NY: The University of Rochester Press.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications. (may be worth paying for if such things interest you).

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Cairo: Ford Foundation. http://www.managingforimpact.org/sites/default/files/resource/outome_harvesting_brief_final_2012-05-2-1.pdf.