Classrooms may one day learn us – but not yet

Thanks to Jim and several others who have recently brought my attention to IBM’s rather grandiose claim that, in a few years, classrooms will learn us. The kinds of technology described in this article are not really very new. They have been just around the corner since the 60s and have been around in quantity since the early 90s when adaptive hypermedia (AH) and intelligent tutoring systems (ITS) rose to prominence, spawning a great many systems, and copious research reported on in hundreds of conferences, books and journal articles. A fair bit of my early work in the late 90s was on applying such things to an open corpus, which is the kind of thing that has blossomed (albeit indirectly) into the recently popular learning analytics movement. Learning analytics systems are essentially very similar to AH systems but mostly leave the adaptation stage of the process up to the learner and/or teacher and tend to focus more on presenting information about the learning process in a useful way than on acting on the results. I’ve maintained more than a passing interest in this area but I remain a little on the edge of the field because my ambitions for such tools have never been to direct the learning process. For me, this has always been about helping people to help one another to learn, not to tell them or advise them on how to learn, because people are, at least till now, the best teachers and an often-wasted resource. This seemed intuitively obvious to me from the start and, as a design pattern, it has served me well. Of late, I have begun to understand better why it works, hence this post.

The general principle behind any adaptive system for learning is that there are learners, some kind of content, and some means of adapting the content to the learners. This implies some kind of learner model and a means of mapping that to the content, although I believe (some disagree) that the learner model can be disembodied in constituent pieces and can even happily exist outside the systems we build, in the heads of learners. Learning analytics systems are generally all about the learner model and not much else, while adaptive systems also need a content model and a means of bringing the two together.  

Beyond some dedicated closed-corpus systems, there are some big obstacles to building effective adaptive systems for learning, or that support the learning process by tracking what we are doing.  It’s not that these are bad ideas in principle – far from it. The problem is more to do with how they are automated and what they automate. Automation is a great idea when it works. If the tasks are very well defined and can be converted into algorithms that won’t need to be changed too much over time, then it can save a lot of effort and let us do things we could not do before, with greater efficiency. If we automate the wrong things, use the wrong data, or get the automation a little wrong, we create at least as many problems as we solve. Learning management systems are a simple case in point: they automated abstracted versions of existing teaching practice, thus making it more likely that existing practices would be continued in an online setting, even though they had in many cases emerged for pragmatic rather than pedagogic reasons that made little sense in an online environment. In fact, the very process of abstraction made this more likely to happen. Worse, we make it very much harder to back out when we automate, because we tend to harden a system, making it less flexible and less resilient. We set in stone what used to be flexible and open. It’s worse still if we centralize that, because then whole systems depend on what we have set in stone and you cannot implement big changes in any area without scrapping the whole thing. If the way we teach is wrong then it is crazy to try to automate it. Again, learning management systems show this in spades, as do many of the more popular xMOOC systems. They automate at least some of the wrong things (e.g. courses, grading, etc). So we had better be mighty sure about what we are automating and why we are doing it. And this is where things begin to look a bit worrying for IBM’s ‘vision’. At the heart of it is the assumption that classrooms, courses, grades and other paraphenalia of educational systems are all good ideas that are worth preserving. The problem here is that these evolved in an ecosystem that made them a sensible set of technologies at the time but that have very little to do with best practice or research into learning. This is not about learning – it is about propping up a poorly adapted system.

If we ignore the surrounding systems and start with a clean slate, then this should be a set of problems about learning. The first problem for learning analytics is to identify what are we should be analyzing, the second is to understand what the data mean and how to process them, the third to decide what to do about that. Our knowledge on all three stages is intermediate at best. There are issues concerning what to capture, what we can dicover about learners through the information we capture, and how we should use that knowledge to help them learn better. Central to all of this is what we actually know about education and what we have discovered works best – not just statistically or anecdotally, but for any and all individuals. Unfortunately, in education, the empirical knowledge we have to base this on is very weak indeed.

So far, the best we can come up with that is fairly generalizable (my favourite example being spaced learning) is typically only relevant to small and trivial learning tasks like memorization or simple skill acquisition. We’re pretty good at figuring out how to teach simple things well, and ITS and AH systems have done a pretty fair job under such circumstances, where goals (seldom learning goals – more often proxies like marks on tests or retention rates) are very clear and/or learning outcomes very simple. As soon as we aim for more complex learning tasks, the vast majority of studies of education are either specific, qualitative and anecdotal, or broad and statistical, or (more often than should be the case) both. Neither is of much value when trying to create an algorithmic teacher, which is the explicit goal of AH and ITS, and is implied in the teaching/learning support systems provided by learning analytics.  

There are many patterns that we do know a lot about, though they don’t help much here.  We know, for example, that one-to-one mastery teaching on average works really brilliantly – Bloom’s 2-sigma challenge still stands, about 30 years after it was first made. One-to-one teaching is not a process that can be replicated algorithmically: it is simply a configuration of people that allows the participants to adapt, interact and exchange or co-develop knowledge with each other more effectively than configurations where there is less direct contact between people.  It lets learners express confusion or enthusiasm as directly as possible, and for the teacher to provide tailored responses, giving full and undistracted attention. It allows teachers to directly care both for the subject and for the student, and to express that caring effectively. It allows targeted teaching to occur, however that teaching might be enacted. It is great for motivation because it ticks all the boxes on what makes us self-motivated. But it is not a process and tells us nothing at all about how best to teach nor how best to learn in any way that can be automated, save that people can, on the whole, be pretty good at both, at least on average.

We also know that social constructivist models can, on average, be effective, for probably related reasons. it can also be a complete disaster. But fans of such approaches wilfully ignore the rather obvious fact that lots of people often learn very well indeed without them – the throwaway ‘on average’ covers a massive range of differences between real people, teachers and learners, and between the same people at different times in different contexts. This shouldn’t come as a surprise because a lot of teaching leads to some learning and most teaching is neither one-to-one nor inspired by social constructivist thinking. Personally, I have learned phenomenal amounts, been inspired and discovered many things through pretty dreadful teaching technologies and processes, including books and lectures and even examined quizzes. Why does it work? Partly because how we are taught is not the same thing at all as how we learn. How you and I learn from the same book is probably completely different in myriad ways. Partly it is because it ain’t what you do to teach but how you do it that makes the biggest difference. We do not yet have an effective algorithmic way of making or even identifying creative and meaningful decisions about what will help people to learn best – it is something that people and only people do well. Teachers can follow an identical course design with identical subject matter and turn it into a pile of junk or a work of art, depending on how they do it, how enthusiastic they are about it, how much eye contact they make, how they phrase it, how they pace it, their intonation, whether they turn to the wall, whether they remembered to shave, whether they stammer etc, etc, etc, and the same differentiators may work sometimes and not work others, may work for some people sometimes and not others. Sometimes, even awful teaching can lead to great learning, if the learners are interested and learn despite rather than because of the teacher, taking things into their own hands because the teaching is so awful. Teaching and learning, beyond simple memory and training tasks, are arts and not sciences. True, some techniques appear to work more often than not (but not always), but there is always a lot of mysterious stuff that is not replicable from one context to the next, save in general patterns and paradigms that are mostly not easily reduced to algorithms. It is over-ambitious to think that we can automate in software something we do not understand well enough to turn into an algorithm. Sure, we learn tricks and techniques, just like any artist, and it is possible to learn to be a good teacher just as it is possible to learn to be a good sculptor, painter or designer. We can learn much of what doesn’t work, and methods for dealing with tricky situations, and even a few rules of thumb to help us to do it better and processes for learning from our mistakes. But, when it comes down to basics, it is a creative process that can be done well, badly or with inspiration, whether we follow rules of thumb or not, and it takes very little training to become proficient. Some of the best teachers I’ve ever known have used the worst techniques. I quite like the emphasis that Alexandra Cristea and others have put on designing good authoring environments for adaptive systems because they then become creative tools rather than ends in themselves, but a good authoring tool has, to date, proved elusive and far too few people are working on this problem.

‘Nothing is less productive than to make more efficient what should not be done at all’. Peter Drucker

The proponents of learning analytics reckon they have an answer to this problem, by simply providing more information, better aggregated and more easily analyzed. It is still a creative and responsive teacher doing the teaching and/or a learner doing learning, so none of the craft or art is lost,  but now they have more information, more complete, more timely, better presented, to help them with the task so that they can do it better. The trouble is that, if the information is about the wrong things, it will be worse than useless. We have very little idea what works in education from a process point of view so we do not know what to collect or how to represent it, unless all we are doing is relying on proxies that are based on an underlying model that we know with absolute certainty is at least partly incorrect or, at best, is massively incomplete. Unless we can get a clearer idea of how education works, we are inevitably going to be making a system that we know to be flawed to be more efficient than it was. Unfortunately, it is not entirely clear where the flaws lie especially as what may be a flaw for one may not be for another, and a flaw in one context may be a positive benefit in another.  When performing analytics or building adaptive systems of any kind, we focus on proxies like grades, attention, time-on-task, and so on – things that we unthinkingly value in the broken system and that mean different things to different people in different contexts.  Peter Drucker made an important observation about this kind of thing:

Nothing is less productive than to make more efficient what should not be done at all‘.

A lot of systems of this nature improve the efficiency of bad ideas. Maybe they valorize behaviourist learning models and/or mediaeval or industrial forms of teaching. Maybe they increase the focus on grading. Maybe they rely on task-focused criteria that ignore deeper connective discoveries. Maybe they contain an implied knowledge model that is based on experts’ views of a subject area, which does not normally equate to the best way to come by that knowledge. Maybe they assume that time on task matters or, just as bad, that less time spent learning means the system is working better (both and neither are true). Maybe they track progress through a system that, at its most basic level, is anti-educational. I have seen all these flaws and then some. The vast majority of tools are doing education-process analytics, not learning analytics. Even those systems that use a more open form of analytics which makes fewer assumptions about what should be measured, using data mining techniques to uncover hidden patterns, typically have risky systemic effects: they afford plentiful opportunities for filter bubbles, path dependencies, Matthew Effects and harmful feedback loops, for example. But there is a more fundamental difficulty for these systems.  Whenever you make a model it is, of necessity, a simplification, and the rules for simplification make a difference. Models are innately biased, but we need them, so the models have to be good. If we don’t know what it is that works in the first place then we cannot have any idea whether the patterns we pick out and use to help people guide their learning journeys are a cause, an effect or a by-product of something else entirely. If we lack an explicit and accurate or useful model in the first place, we could just again be making something more efficient that should never be done at all. This is not to suggest that we should abandon the effort, because it might be a step to finding a better model, but it does suggest we should treat all findings gathered this way with extreme scepticism and care, as steps towards a model rather than an end in themselves.

In conclusion, from a computing perspective, we don’t really know much about what to measure, we don’t have great grounds for deciding how to process what we have measured, and we don’t know much at all about how to respond to what we have processed. Real teachers and learners know this kind of thing and can make sense of the complexity because we don’t just rely on algorithms to think. Well, OK, that’s not necessarily entirely true, but the algorithms are likely at a neural network level as well as an abstract level and are probably combinatorially complex in ways we are not likely to understand for quite a while yet. It’s thus a little early to be predicting a new generation of education. But it’s a fascinating area to research that is full of opportunities to improve things, albeit with one important proviso: we should not be entrusting a significant amount of our learning to such systems just yet, at least not on a massive scale. If we do use them, it should be piecemeal and we should try diverse systems rather than centralizing or standardizing in ways that the likes of Knewton are trying to do. It’s bit like putting a computer in charge of decisions whether or not to launch nuclear missiles. If the computer were amazingly smart, reliable and bug-free, in a way that no existing computer even approaches, it might make sense. If not, if we do not understand all the processes and ramifications of decisions that have to be made along the way, including ways to avoid mistakes, accidents and errors, it might be better to wait. If we cannot wait, then using a lot of different systems and judging their different outputs carefully might be a decent compromise. Either way, adaptive teaching and learning systems are undoubtedly a great idea, but they are, have long been, and should remain on the fringes until we have a much clearer idea of what they are supposed to be doing. 

Being-taught habits vs learning styles

In case the news has not got through to anyone yet, research into learning styles is pointless. The research that proves this is legion but, for instance, see (for just a tiny sample of the copious and damning evidence):

Riener, C., & Willingham, D. (2010). The Myth of Learning Styles. Change: The Magazine of Higher Learning Change: The Magazine of Higher Learning, 42(5), 32-35. doi:doi: 10.1080/00091383.2010.503139

Derribo, M. H., & Howard, K. (2007). Advice about the use of learning styles: A major myth in education. Journal of college reading and learning, 37, 2.

Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning: A systematic and critical review. 041543).

No one denies that it is possible to classify people in all sorts of ways with regards to things that might affect how they learn, nor that everyone is different, nor that there are some similarities and commonalities between how people prefer to or habitually go about learning. When these elaborately constructed theories claim no more than that people are different in interesting and sometimes identifiably consistent ways, then I have little difficult accepting them in principle, though it’s always worth observing that there are well over 100 of these theories and they cannot all be right. There is typically almost nothing in any of them that could prove them wrong either. This is a hallmark of pseudo-science and should set our critical sensors on full alert. The problem comes when the acolytes of whatever nonsense model is their preferred flavour try to take the next step and tell us that this means we should teach people in particular ways to match their particular learning styles. There is absolutelly no plausible evidence that knowing someone’s learning style, however it is measured, should have any influence whatsoever on how we should teach them, apart from the obvious requirement that we should cater for diversity and provide multiple paths to success. None. This is despite many decades spent trying to prove that it makes a difference. It doesn’t.

It is consequently a continual source of amazement to me when people pipe up in conversations to say that we should consider student learning styles when designing courses and learning activities. Balderdash. There is a weak case to be made that, like astrology (exactly like astrology), such theories serve a useful purpose of encouraging people to reflect on what they do and how they behave. They remind teachers to consider the possibility that there might be more than one way to learn something and so they are more likely to produce useful learning experiences that cater for diverse needs, to try different things and build flexibility into their teaching. Great – I have no objection to that at all, it’s what we should be aiming for. But it would be a lot more efficient to simply remind people of that simple and obvious fact rather than to sink vast sums of money and human resources into perpetuating these foolish myths. And there is a darker side to this. If we tell people that they are (just a random choice) ‘visual’, or  ‘sensing’ or ‘intuitive’ or ‘sequential’ learners then they will inevitably be discouraged from taking different approaches. If we teach them in a way that we think fits a mythical need, we do not teach them in other ways. This is harmful. It is designed to put learners in a filter bubble. The worst of it is that learners then start to believe it themselves and ignore or undervalue other ways of learning.

Being-taught habits

The occasion for this rant came up in a meeting yesterday, where it was revealed that a surprising number of our students describe their learning style (by which they actually mean their learning preference) to be to listen to a video lecture. I’m not sure where to begin with that. I would have been flabbergasted had I not heard similar things before. Even learning style believers would have trouble with that one. One of the main things that is worth noting, however, is that this is actually a description not of a learning preference but of a ‘being-taught habit’. Not as catchy, but that’s what it is.

I have spent much of my teaching career not so much teaching as unteaching: trying to break the appalling habits that our institutional education systems beat into us until we come to believe that the way we are being taught is actually a good way to learn. This is seldom the case – on the whole, educational systems have to achieve a compromise between cost-efficiency and effective teaching –  but, luckily, people are often smart enough to learn despite poor teaching systems. Indeed, sometimes, people learn because of poor teaching systems, inasmuch as (if they are interested and have not had the passion sucked out of them) they have to find alternative ways to learn, and so become more motivated and more experienced in the process of learning itself. Indeed, problem-based and enquiry-based techniques (which are in principle a good idea) sometimes intentionally make use of that kind of dynamic, albeit usually with a design that supports it and offers help and guidance where needed.

If nothing else, one of the primary functions of an educational system should be to enable people to become self-directed, capable lifelong learners. Learning the stuff itself and gaining competence in a subject area or skill in doing something is part of that – we need foundations on which to build. But it is at least as much about learning ways of learning. There are many many ways to learn, and different ways work better for different people learning different things. We need to be able to choose from a good toolkit and use approaches that work for the job in hand, not that match the demands of some pseudo-scientific claptrap.

Rant over.

 

Teaching gestalts

I’m preparing for a presentation and discussion tomorrow with some doctoral students on the orchestration of lifelong learning. Having come up with the topic some time ago on a whim I’m not entirely sure what I’ll be talking about, so this is mostly an attempt to focus my thinking a little and is very much a work in progress.

In brief, the central jumping off point for this discussion is that teachers are not isolated actors but are instead are gestalts formed from

  • numerous technologies, including pedagogies, regulations, processes, techniques and tools,
  • an uncountably large number of individuals and groups and, most notably of all,
  • learners themselves.

For it to work, everything must harmonize or must make the right kinds of dischord to bring about learning. There are various things that shake out of this perpsective, not least of which being that there are many ways to organize this teaching gestalt that do not involve an educational system of the sort we are used to, and that do not involve individuals labelled as teachers. This matters because most of the learning we do throughout our lives does not take place in or result from formal education.

The teaching gestalt

Even and perhaps particularly in a traditional educational system, teachers are not just the ones that stand (metaphorically or actually) in front of classes and explicitly perform an act that we label as teaching. Teachers are also the authors, editors, illustrators, designers and publishers of textbooks, the builders of websites, the writers of articles and so on. Teachers are designers of school systems, timetablers, architects, designers and furniture builders. Teachers are makers of videos, programmers of online environments, system administrators, TV producers, designers of door handles and technicians. And, above all, learners are teachers – of themselves and of one another. In short, teaching is always a distributed role.

Unpicking this a little further, almost all learning transactions involve at least two teachers – the one with knowledge of content, process, etc, and the learner. Learning is always an active process of knowledge construction, linking, and sense-making in which we constantly reflect, reorientate, examine, and adjust our knowledge in the light of new information or new ways of seeing. We always teach ourselves at least as much as we are taught. We are not given knowledge – we make it. Another person may help to guide us, shape the directions we go, correct us when we are confused or wrong, and motivate us to go the extra mile, but we are always a teacher in this process, whether we like it or not.

In an educational context, a vast array of actors add their own contributions to the teaching whole. Some, like authors of textbooks, or creators of curricula, or other students sharing ideas and (mis)conceptions are very obviously playing a teaching role. Others are less obviously so, but they do matter. The people that made decisions about where to place a whiteboard, which tools to enable in an LMS, or what wattage of lightbulb to include in a classroom may make a huge contribution to the success of failure of a particular learning transaction. The designer of the timetable, the legislator who demanded a particular kind of content or a particular kind of behaviour, the setter of normalized tests, the curriculum designer and the person who cleaned the classroom, all play significant and sometimes crucial roles as part of the teaching gestalt. Timetables teach, LMSs teach, hallways teach. In an educational system it is the system that educates, not just the individual teacher. I particularly like the timetable example because it is a great rejoinder to those who rather naively suggest that teachers should put pedagogy first. Sure: but first you must do it only at these times, over this period, for this amount of time, in this physical or virtual place, on this subject. Whatever. Anyway, within this context, the person who is performing the explicit role of a teacher is thus just one of the teaching gestalt but, potentially, quite a special one, sometimes (but not always) second only to the learner in importance. He or she typically acts as a filter, conduit and interpreter that orchestrates this whole, that responds, gives feedback, shows caring. It’s not too surprising that we label this person differently from the rest of the gestalt.

Orchestral manoeuvring

Since we are talking about a process of orchestration, it is natural to think of music at this point, and the analogy works quite well. A teacher may be an orchestrator, adapting to a context in which many constraints and structures have already been determined by others, using the tools, techniques and technologies to play a part in the construction of knowledge that is hopefully the outcome. Some are conductors, trying to elicit harmonious learning through tight control of the process. Like the best conductors, the best teachers of this sort make use of the materials they are working with, fitting the strengths and weaknesses of the players, the acoustics of the venue, the nature of the instruments, to the demands of the piece to be played and the intended audience. Other teachers are more like arrangers, who organize the pieces and leave the playing to someone else. Some are like players in a band, maybe drummers or bassists providing a rhythm to keep learners on track, or perhaps as soloists showing virtuosity and improvisational skills that inspire the learners to new heights. Some are content to play second fiddle, bringing out the best in the soloist but always in the background. And then there are the ones who sit in a recording studio who play all the instruments themselves, sometimes even making the instruments, and arrange everything the way they want it to be arranged. Some play blues, using the same three chords and often simple technique to play an infinite and subtle range of tunes. Some play classically, sticking closely to but always interpreting a score. Some are composers. Some are jazz improvisors, modern or trad. Some go for unusual scales, exotic rhythms and peculiar blends, others prefer the folk traditions that they learned as children. The sounds that musicians make are a function of many things, including most notably the instrument itself as well as the surroundings in which it is played and the reactions of an audience. And, in most cases, there are many instruments to consider. A lot of the process of teaching is about the technologies tools and techniques, incredibly diverse, all of which have to work to a common purpose.

But whatever the tools, genres, blends and roles that teachers play, when it comes down to basics, teachers (that is to say, the players in the teaching gestalt) have to be skilled and creative, whatever and however they try to play. Above all, teaching (emerging from all the many contributors to that role) is a broad set of human practices, not a science, not just a set of techniques. It is, moreover, a creative, active and inventive practice that cannot be emptied of soul and programmed into a machine without losing the vitality and expression that makes it wonderful. This is not to suggest that machines cannot or should not be a big part of the process, however, any more than that an orchestra should try to play without instruments or a venue. Putting aside more blatant technologies like classrooms and LMSs, for better or worse, our educational systems are machines that, depending on your perspective and the aspect you are looking at, either enable or disable our ability to learn. Likewise, Google Search and Wikipedia (my two favourite e-learning technologies) have a very large and conspicuous machine element. And, of course, the creativity and inspiration can be distributed too. A bad teacher can be saved by a good textbook, for instance, and vice versa.

Why bother with teachers anyway?

It is tempting to say that most of the intentional learning we do is self-guided – that we teach ourselves anything from cooking to philosophy. I know it’s tempting, because I’ve been known to say it, and have read many research studies purporting to show this. However, this is nearly always massively wrong. What we actually do, in almost all cases, is to orchestrate teaching done by others. In some cases this is blatant and obvious. If we learn something by reading a Wikipedia article, or a book, or by watching a video, this is very clearly not a case of us teaching ourselves. At least, not totally. We are merely picking our teachers and exercising a bit of control over the pace, time and place that they teach us. We don’t get all the benefits of teaching that way by any means – importantly, we seldom get much in the way of feedback, for example, and any tailoring that happens is up to us. These kinds of things do not show us that they care about us. Such things are co-teachers, part of the teaching gestalt. But it is all a matter of degree: we are always our own teachers to some extent, and there are almost always others involved in teaching us, no matter how informal or formal the setting. Even when we learn by dabbling and experimenting, we are not exactly pure autodidacts. Partly this is because we often have some kind of target to aspire to because we have seen, read, heard or otherwise encountered terminal behaviours of the sort we are aiming for. For many competences, it is because the things we try to learn or learn with are typically designed by humans who have other humans in mind when they design them – this is true of learning that makes use of things like pencils, paints, cookware, computers, cars, musical instruments, exercise machines, calculators and yachts.  Learning in a vacuum is not possible, unless we are learning about the vacuum which might be, incidentally, one of those rare occasions where no other teacher is directly involved in the process.

By way of example, in recent years,  I have been ‘teaching myself’ to play a new instrument at least once a year. I know what these instruments sound like when they are played well, so I can recognize the gaps between what I can do with them and what they can do. Many teachers have taught me. I have seen other people playing them so I have a fair idea how to hold them but, on the whole, they are designed to be held and manipulated so it seldom takes too long to figure that out by trial and error. Their designers have taught me. That said, I challenge anyone to watch someone else play the flute and, based on what you get out of that, to make the flute sound the same. It’s mighty hard. You might get the odd note and you might even figure out how to shape your mouth differently to switch octaves, but simply copying is probably not quite enough. Most instruments have quirks like that and it would not normally be very wise to simply rely on trial and error. The actual process I generally follow usually involves reading a bit about fingerings, tunings, breathing, embouchure and so on, usually with instrument in hand so that I can check what it all means, then a lot of trial and error, lots of YouTube videos and a great deal of practice until I reach a plateau, after which the cycle repeats again as I learn how to do more advanced stuff like overtones, harmonics, complex chords, intonation, picking or bowing styles, etc. I am never going to become a virtuoso this way, sure, but it is loosely structured in a way that leads to a bit more than the outcome of a chopsticks culture (this refers to Alan Kay’s delightful analogy of what happens when you simply put a computer in a classroom and hope for the best). Eventually I need to play with other people who play better or differently, to get a bit of coaching, to find others who will challenge me to go beyond my comfort zone, but I generally wind up being competent to carry a tune reasonably enough before getting to that point. Part of the reason that I can do this kind of thing because I have learned to teach myself and, of course, I am building on a foundation of existing knowledge. I can read music. I’ve grappled with most families of musical instrument at some point. I know the difference between 3/4 and 4/4 time, and a little bit about harmony. And I know a little about how people learn. All of this is because I have had many teachers, very few of whom were intentionally playing that role.

The unsaid

This all leads to what will, in my talk tomorrow, be the jumping off point for the real discussion, and some questions to which I have some answers but mostly not the best ones. What do all the things that go up to make teachers actually do?  What is the value professional teachers add? How can we manage our teachers? How can we replace them? As professional teachers, how can we allow our students to manage us? What aspects of educational systems teach? What alternative ways of organizing and orchestrating learning might we discover, invent or adapt? I’m particularly interested in exploring ways to overcome some of the manifestly awful teaching that our educational systems do to our students like grading, for instance, and what to do when the tunes we want to play are not in harmony with those played by the systems we are working in. But I am also interested in exploring ways that we can enable people to be better orchestrators of their own inner and outer teachers, beyond institutional contexts, beyond xMOOCs, beyond simple tutorials. I’m hoping it will be a fun discussion. How best to characterize what I’m aiming for? A bit of jazz improvisation, perhaps.

 

MOOPhD accreditation

A recent post at http://www.insidehighered.com/views/2013/06/05/essay-two-recent-discussions-massive-open-online-education reminded me that the half-formed plan that Torsten Reiners, Lincoln Wood and I dreamt up needs a bit of work.

So, to add a little kindling to get this fire burning…

Our initial ideas centred around supporting the process of doing research and writing papers for a PhD by publication. This makes sense and, we have learned, PhDs by publication are actually the norm in many countries, including Sweden, Malaysia and elsewhere, so it is, in principle, do-able and does not require us to think more than incidentally about the process of accreditation. However, there are often invisible or visible obstacles that institutions put in place to limit the flow of PhDs by publication: residency requirements, only allowing them for existing staff, high costs, and so on.

So why stop there?

Cranking the levers of this idea pump a little further, a mischievous thought occurs to me. Why not get a PhD on reputation alone? That is, after all, exactly how any doctorate is awarded, when it comes down to it: it is basically a means of using transferable reputation (think of this as more like a disease than a gift – reputations are non-rival goods), passing it on from an institution to an awardee, with a mutational process built in whereby the institution itself gets its own research reputation enhanced by a similar pass-it-on process. This system honours the institution at least as much as the awardee, so there’s a rich interchange of honour going on here. Universities are granted the right to award PhDs, typically through a government mandate, but they sustain their reputation and capacity to do so through ongoing scholarship, publication and related activities, and through the activities of those that it honours. A university that awarded PhDs without itself being a significant producer of research, or that produced doctors who never achieved any further research of any note, would not get very far. So, a PhD is only a signal of the research competence in its holder because an awarding body with a high reputation believes the holder to be competent, and it sustains its own reputation through the activities of its members and alumni. That reputation occurs because of the existence of a network of peers, and the network has, till now, mostly been linked through journals, conferences and funding bodies. In other words, though someone goes to the trouble of aggregating the data, the actual vector of reputation transmission is through individuals and teams that are linked via a publication process. 

So why not skip the middle man? What if you could get a PhD based on the direct measures of reputation that are currently aggregated at an institutional level rather than those that have been intentionally formalized and aggregated using conventional methods?

Unpicking this a little further, the fact that someone has had papers published in journals implies that they have undergone the ordeal by fire of peer review, which should mean they are of doctoral quality. But that doesn’t mean they are any good. Journals are far from equal in their acceptance rates, the quality of their reviewers – there are those with good reputations, those with bad ones, and a lot in between. Citations by others help to assure us that they may have something of value in them, but citations often come as a result of criticism, and do not imply approval of the source. We need a means to gauge quality more accurately. That’s why h-index was invented. There are lots of reasons to be critical of this and similar measures: they fail to value great contributions (Einstein would have had a very low h-index had he only published his most important contributions), they embody the Matthew Effect in ways that make their real value questionable,  they poorly distinguish large and small contributions to collaborative papers, and the way they rank importance of journals etc is positively mediaeval. It is remarkable to me to surf through Google Scholar’s rankings and find that people who are among the most respected in my field having relatively low indexes while those that just plug away at good but mundane research having higher ones. Such indexes do none-the-less imply the positive judgements of many peers with more rigour and fairness than would normally be found in a doctoral committees, and they give a usable number to grade contributions. So, a high h-index or i10-index (Google’s measure of papers with more than 10 citations) would satisfy at least part of the need for validation of quality of research output. But, by definition, they undervalue the work of new researchers so they would be poor discriminators if they were the only means to evaluate most doctorates. On the other hand, funding councils have already developed fairly mature processes for evaluating early-career researchers, so perhaps some use could be made of those. Indeed, the fact that someone has successfully gained funding from such a council might be used as partial evidence towards accreditation.

A PhD, even one by publication, is more than just an assortment of papers. It is supposed to show a sustained research program and an original contribution to knowledge. I hope that there are few institutions that would award a PhD to someone who had simply had a few unrelated papers published over a period of years, or to someone who had done a lot of mundane but widely cited reports with no particular research merit. So, we need a bit more than citation indexes or other evidence of being a world-class researcher to offer a credible PhD-standard alternative form of certification.

One way to do this would be to broadly mirror the PhD by publication process within the MOOC. We could require peer ‘marking’, by a suitable panel, of a paper linking a range of others into a coherent bit of doctoral research and perhaps defended in a public webmeeting. This would be a little like common European defence processes, in which theses are defended not just in front of professors but also any member of the public (typically colleagues, friends and families) who would want to come along. We could increase the rigour a little by making it a requirement that those participating in such a panel should have to have a sufficiently high h-index or i-index of their own in a similar subject area, and/or have a relevant doctorate. Eventually the system could become self-supporting, once a few graduates had emerged. In time, being part of such a panel would become a mark of prestige in itself. Perhaps, for pedagogic and systemic reasons, engagement in such a panel would be a prerequisite for making your own ‘doctoral’ defence. Your rating might carry a weighting that accorded with your own reputational index, with those starting out weighted quite low and those with doctorates, ‘real’ doctoral students etc having higher indexes. The candidates themselves and other more experienced examiners might rate these novice examiners, so a great review from an early-career candidate might increase their own ranking.  It might be possible to make use of OpenBadges for this, with badges carrying different weights according to who awarded them and for what they were awarded.

Apart from issues of motivation, the big problem with the peer-based approach is that it could be seen as one of the blind leading the blind, as well as potentially raising ethical issues in terms of bias and lack of accountability. A ‘real’ PhD committee/panel/etc is made up of carefully chosen gurus with an established reputation or, at least, it should be. In North America these are normally the people that supervise the student, which is dodgy, but which normally works OK due to accountability and professional ethics. Elsewhere examiners are external and deliberately unconnected with the candidate, or consist of a mix of supervisors and externals. Whatever the details, the main point here is that the examiners are fully accredited experts, chosen and vetted by the institutional processes that make universities reliable judges in the first place. So, to make it more accountable, more use needs to be made of that reputational network that sustains traditional institutions, at least at the start. To make this work, we would need to get a lot of existing academics with the relevant skills on board. Once it had been rolling for a few years, it ought to become self-sustaining.

This is just the germ of an idea – there’s lots of ways we could build a very cheap system that would have at least as much validity as the accreditation procedures used by most universities. If I were an employer, I’d be a lot more impressed by someone with such a qualification than I would by someone with a PhD from most universities. But I’m just playing with ideas here. My intent is not to create an alternative to the educational system, though that would be very interesting and I don’t object to the idea at all, but to highlight the often weird assumptions on which our educational systems are based and ask some hard questions about them. Why and on what grounds do we set ourselves up as arbiters of competence? What value do we actually add to the process? How, given propensities of new technologies and techniques, could we do it better? 

Our educational systems are not broken at all: they are actually designed not to work. Well, ‘design’ is too strong a word as it suggests a central decision-making process has led to them, whereas they are mainly the result of many interconnected decisions (most of which made sense at the time but, in aggregate, result in strange outcomes) that stretch back to mediaeval times. Things like MOOCs (and related learning tools like Wikipedia, the Khan Academy, StackOverflow, etc) provide a good opportunity to think more clearly and concretely about how we can do it better and why we do it the way we do in the first place.

Unintelligent design and the modern MOOC

Everyone is talking about MOOCs.

Every institution of higher learning I visit or talk with seems intent on joining the MOOC scrum or, if not, is coming up with arguments why it shouldn’t. There’s also a wealth of poorly considered, badly researched opinion pieces too, many of them published by otherwise fairly reputable journals and news sources. I’ve been doing my bit to add poorly researched opinion too, talking in various venues about a few ideas and opinions that are not sufficiently rigorously explored to make into a decent paper. This post is still not worthy of a paper, but I think the main idea in it is worth sharing anyway. To save you the trouble of reading the whole thing, I’m going to be making the point that MOOCs disrupt because they quietly remove two of the almost-never-questioned but most-totally-nonsensical foundations on which most traditional university teaching is based – integral accreditation and fixed course lengths – and their poor completion rates therefore encourage us/force us to ask ourselves why we do such things. My hope is that the result of such reflection will be to bring about change. To situate my opinions relative to those of others, I will start by offering a slight caricature of the three main stances that people seem to be taking on MOOCs.

Opinion 1 – it’s all rubbish and online learning is pants

The cantankerati are, of course, telling us that there is nothing new here, or that online learning isn’t as good as face to face, or that it is all hype, or that the learning outcomes are not as good as those at (insert preferred institution, preferably one’s alma mater, here) etc. This is a fad, they tell us. They look at things like drop-out rates or Udacity partnering with Georgia Tech or Coursera moving into competition with Blackboard, or the fact that millenial college students prefer traditional to online classes (err – seriously? that’s like asking iPhone users if they prefer them to Android phones) and nod their heads sagely, smugly and in an ‘I told you so’ fashion. No doubt, when the bubble bursts (as it will) they will be the first to gloat. But they are wrong about the failings of MOOCs, on most significant counts.

Opinion 2 – it’s a step in the right direction, but (insert prejudice here)

Others think that there is something worth preserving here and are trying to invent new variants – usually xOOCs of some kind, or MOOxs, or, in rare cases, xOOxs, liking some aspect of the MOOC idea such as openness or size but not liking others. The acolytes of online learning (AOLs for short, oddly enough) are getting all excited about the fact that people are at last paying attention to what they have been saying for years, though most are tempering their enthusiasm with observations about the appalling pedagogies, the creation of a two-tier system of higher education, problems with accrediting MOOC learning, and  high ‘dropout’ rates. They are wondering why these MOOCish upstarts haven’t read their own august works on the subject which would obviously steer them right.  They will, when pressed, grudgingly admit that these rank enthusiastic amateurs are (dammit) quite signally succeeding in ways they have only dreamed of, but they still know better. There are many of these,  some of which are actually very thoughtful and penetrating and by no means unsubtle in their analysis:  John Daniel’s well-informed sagacious overviewPaul Stacey’s intelligent mourning of the overshadowing of a good idea, or Carol Edwards’s slightly jaundiced but interesting and revealing first-person report for BCIT, for instance. There are far more unsubtle and far less well-informed rants that I won’t bother linking here that complain about the pedagogies, or tell us that there is nothing new at all in this, or that think they see an alternative future etc. Oh, alright – here’s one that I find particularly silly and here are my comments on it.

Opinion 3 – the sky is falling! The sky is falling!

There is a third group that is fairly sure that MOOCs are very important and that they are causing or, at least, catalyzing a seismic shift in education. The popular press clearly demonstrates that there’s a revolution happening, for better or worse, and most people who hold this position want to be on that bandwagon, wherever it may be going. If not, they fear they will be left in the dust. There are some notable holders of this perspective who justify and examine their beliefs in intelligent ways, such as the ever-brilliant Donald Clark, for example, who has recently written a great series of posts that are both critical and rabble-rousing.

And many in between…

Between and spanning these caricatures are some really interesting and perceptive commentaries, and only a few have as clear-cut an opinion as I portray here. Aaron Bady’s post casting a critical eye on the hype, for example, picks apart the sky falling very carefully, and situates itself a little in the ‘right direction’ camp without being too much on the ‘but…’ side of things. The recent Edinburgh report on their pilot MOOCs is a model of careful research and openness to critical and creative thinking.   George Siemens’s excellent analysis of x-vs-c MOOCs is another great piece that avoids much bias one way or the other while identifying some of the key issues for the future.

Where I sit

You could call me a fan. My PhD (completed well over 10 years ago) was largely about how large online crowds can learn together. I’ve signed up for (but not completed) quite a few MOOCs since 2008, and I’ve been a more active participant at times, playing a teaching role in a couple and helping to lead one in early 2011. I ran my first education-oriented web server offering what we would now call open educational resources in 1993. I read an average of two or three articles on MOOCs every day, maybe more. I’ve joined up with the newly formed WideWorldEd project and have been engaged in discussions and planning about MOOCs at three different institutions.

I am definitely not one of the cantakerati though I am highly sceptical of any blanket claim that a particular flavour of teaching leads to better or worse learning than any other, be it online or not. It ain’t what you do, it’s the way that you do it.

I do not believe that the pedagogies of most MOOCs are particularly bad or retrograde. Talking heads, objective tests and other favourite tools of early xMOOC providers are not my cup of tea, and the chaos of cMOOCs (that I like a lot more) seems to favour only a few neterate winners, but most that I have seen are actually at least as good as their paid-for counterparts. There are quite a lot that do not fall neatly into either of these main camps too – e.g. http://ds106.us – and both camps share a lot in common with each other that neither camp seems particularly happy to acknowledge: connectivist networks thread through and around xMOOCs and disrupt their neat outlines, while cMOOCs often employ what look and smell a lot like instructivist lectures as significant parts of the process. But, whatever the similarities, what and how people teach is seldom what and how people actually learn so it is not that important. Quality is not a direct correlate of the pedagogies and other technologies used. In fact, it is interesting to note that a recent article on MOOC junkies highlighted the greater significance of passion in the professor, something I and many others have been saying for quite a while. It ain’t what you do, it’s the way that you do it.

For me, the sky is not falling yet though it certainly has a few more interesting colours than it had a year or two ago and there are some fascinating systemic effects that are mostly, but not all, positive. But this is not the beginning of the end of higher education as we know it. In some ways, it could be the beginning of  something much more interesting.

What really appeals to me most about MOOCs is their almost universally low completion rates. Whatever this means for MOOCs themselves, and however much it upsets their providers (not their learners), in my opinion this is by far their most positive systemic feature. While It ain’t what you do, it’s the way that you do it, I have one important proviso that needs to be added to that: there are some things that you can do that will most probably and in some cases definitely fail to get results. And this is really what this post is about.

So, what about those completion rates?

One thing that many of the cantankerati, the fearfully curious and the AOLs amicably agree on is that that fact that most people drop out of most MOOCs shows that there is something wrong with the idea, or how it has been implemented, or both. Some MOOCs struggle to keep 2% of their students while the best (on horse feeding, as it happens) have managed a little over 40%. The vast majority (so far) have succeeded in keeping less than 10% of their students to the bitter end. This is particularly odd given that, on most MOOCs, the majority of course-takers have at least one degree, many are educators, and quite a few have post-graduate qualifications. These are, for the most part, mature learners who know how to learn and probably think about how they do it.

For some, this is proof that online learning doesn’t work (self-evidently wrong, I’m glad to say, or I and hundreds of thousands of others would be out of a job, Wikipedia would vanish and Google Search would be largely abandoned). For others, it is proof that the pedagogies don’t work (not entirely right either, or no one would take them). The more informed, also known as those who think about it for more than two seconds, realize pretty quickly that MOOCs do not require any strong interest, let alone any significant commitment to sign up to, nor do they demand any prerequisites. So, of course, most people ‘drop out’ within the first couple of weeks, if indeed they pay any attention at all beyond spending less than a minute signing up and vaguely thinking that it might be interesting to take part. They may have insufficient interest, they may find it too hard, too easy, too boring, or too engrossing and demanding of their time. Maybe they don’t like the professor. Maybe they have better things to do. Nor is it any surprise that people whose only commitment is time might drop out after the first couple of weeks – many get what they came for and stop, or they lose interest, or get distracted, or break their computers, or simply run out of time to keep working on it. There has been a little good research and a lot of useful speculation on this, for instance at http://www.katyjordan.com/MOOCproject.html and http://blogs.kqed.org/mindshift/2013/04/why-do-students-enroll-in-but-dont-complete-mooc-courses/ and http://www.openculture.com/2013/04/10_reasons_you_didnt_complete_a_mooc.html and http://mfeldstein.com/emerging_student_patterns_in_moocs_graphical_view/ and http://donaldclarkplanb.blogspot.ca/2013/01/moocs-dropout-category-mistake-look-at.html

But there is something odder going on here that seems to be mostly slipping under the radar, apart from the odd mention here and there by people like Alan Levine and a few others.  I’ve long been bothered by the mysterious and improbable fact that, in higher education, all learning is neatly divisible into 13 (or 15, or 10, or something in that region) week chunks. This normally equates to an average of around 100 hours of study time, give or take a bit. Whatever the particular length chosen, they are almost always unaccountably multiples of chunks of the same size at any given institution, and that size is broadly comparable to other courses/modules/papers/units/etc in other institutions. It’s enough to make you wonder whether there might be a god as it suggests intelligent design may be at work here.

Actually, it’s the result of unintelligent design. This is an evolutionary process in which path dependencies pile up and push their way into adjacent possibles.

So, why do we have courses (or modules/papers/units/etc depending on your geographical region)?

Well, in the first place, it is true that some things take longer to learn than others. Not everything can be mastered by asking a question or looking it up on Wikipedia. That’s completely fair and reasonable. It doesn’t, however, explain why it takes the same amount of time (or multiples of it) for everyone, regardless of skill, experience or engagement, to master everything – Modern European Philosophy, Chemistry 101, Java Data Structures, Literary Culture & the Enlightenment, Icelandic Politics: all fit the same evenly sized periods, or multiples of them. For an explanation of that, we have to turn to a combination of harvest schedules, Christian holidays and the complexities of managing scarce physical resources that are bound by physics to a single and somewhat constrained teaching space.

The word ‘lecturer’ derives from the fact that lecturers used to read from the very valuable and scarce single copies of books held by institutions. Lecture theatres and classrooms were thus the most efficient way to get the content of books heard by the largest possible number of people. If you want to get a lot of people to listen at once then it helps if they are actually there so, if they are taking a religious holiday or helping with the harvest (this last point is a little contentious as it doesn’t fully explain a long break from July to October), there is no point in standing up and talking to an empty lecture hall. So, putting aside Easter’s irritating habit of moving around from year to year that continues to mess up university teaching schedules, this divides things up quite neatly into roughly 13 week chunks separating harvest, Christmas, and Easter breaks. The period may vary a little, but the principle is the same.

This pattern has become quite deeply set into how learning happens at most universities, even though the original reasons it occurred might have faded into insignificance had they not become firmly embedded through momentum and the power of path dependencies. Assessment became intimately linked to the schedule, with ‘mid-terms’ and ‘finals’ and then came to act as a major driver in its own right. Teacher pay and time was allocated according to easily managed chunks and resources. Enrolments, registrations, convocations and the familiar rhythms of the university calendar helped to consolidate the pattern, largely driven by a need for efficiency and bureaucratic convenience. It is really hard to allocate teachers and students to rooms. Up to this point, there was no particular reason to divide the learning experience into modularized chunks and many universities did (and some still do) simply have programs (or programmes or, to confuse matters, courses lasting 3-5 years) with perhaps a few streams but without distinct modularized elements. To cap it off and set it in stone, three forces coincided. One was a laudable desire to allow students the flexibility to take some control over what they learned.  Another was the need to simplify the administration of programs. The last was the need to assert equivalence between what is taught at institutions, whether for certification purposes or for credit transfer. This last force, in particular, has meant that this way of dividing learning into modular chunks of a similar length has become a worldwide phenomenon, even in countries for which Easter and Christmas have no meaning or value.

All of this happened because there had to be a means of managing scarce resources shared among many co-present people as efficiently as possible but, for centuries, there has been no good reason for picking this particular term-length apart from the force of technological momentum.  There have been innovations, here and there. Athabasca University, for instance, gives undergraduates 6 months (extendible at a price) in which to complete work in any way and timeframe that will fit their needs. Similarly, the University of Brighton runs ‘short fat’ masters modules that last for half a week, combined with a period of self-study before and after. But, in order to maintain accreditation parity, the amount of work expected of students on such courses broadly equates to what, in conventional classes, would take – yes – 13-15 weeks. Technically, thanks to a bit of reverse engineering, this translates into roughly 100 hours of study in the UK, a little more or less elsewhere, particularly where people take the insanely bad North American approach of counting teaching hours rather than study hours (what madness gripped people that made them think that was a good idea?).  Whatever the rationale, this has nothing to do with learning, nothing to do with the nature of a topic or subject area, nothing to do with the best way to teach. It’s just the way it turned out, and certification requirements reinforce that anti-educational trend.

So what?

Courses are not neutral technologies. One of the least loveable things about them is that their content, form and process are, at least ostensibly, controlled by teachers from start to finish. Courses are a power trip for educators that, in institutional incarnations, often require some quite unpleasant measures to maintain control, typically based on long-discredited models of human psychology that rely heavily on rewards and punishments – grades, attendance requirements, behavioural constraints in classrooms, etc. That is just plain stupid if you actually want people to learn and believe that it is your job to help that process. There can be few methods apart from deliberate torture and punishment that more reliably take motivated, enthusiastic learners and sap the desire to learn from them. We do this because courses are a certain length and we think that students have to engage in the whole thing or not at all.

Students, meanwhile, have little choice but to accept this or to drop out of the system, but that’s tricky because those uniform-size credentials have become the currency for gaining career advancement and getting a job in the first place.

Teachers need to work on maintaining that control because there are very few topics that can, in and of themselves, sustain a large number of individuals’ interest for 13 solid weeks and those that do are highly unlikely to naturally fit into that precise timeframe. Sure, some students may passionately love the whole thing and may have learned to gain some immunity from the demotivating madness of it all, or the teacher may be one of those rare inspiring people that enthuses everyone she gets to teach. But, for most students, it will be, at best, a mixed bag. Even for those that enjoy much of it, some will be irrelevant, some too easy, some too complicated, some simply dull. But they have to do it because that what the teacher demands that they have to do, and teachers have to fit their courses to this absurd length limit because that’s what the institutions demand that they have to do, and institutions do it because that’s how it has always been done and everyone else does it.

This is not logical.

So much of what makes a great teacher is therefore the ability to overcome insanely stacked odds and work the system so that at least a fair number of people get something good out of it. Teachers have to find ways to enthuse and motivate, to design assessments that are constructively aligned, to perform magic tricks that limit the damage of grading, to build flexible activities that provide learners with a bit of self-determination and control. Sadly, many do not even do that, relying on this juggernaut and the whole unwieldy process to crush students into submission (of assignments). It really doesn’t have to work like that.

This systemic failure is tragic, but understandable and forgivable. There is massive momentum here and opposition to change is designed into the system. It would take a brave teacher to explain to administrators and examination boards that she has decided that the topic she is teaching actually only needs 4 weeks to teach. Or 33 weeks. Or whatever. And, no, it will not have any parity with other courses on the same subject: OK? I would not relish that fight. It is considerably more tragic and less easy to forgive when, without any of those constraints – no formal accreditation, no institutional timetables, no harvest, no regulations, no scarcity of resources  – a few MOOC purveyors do the same thing. What is going on in their heads? My sense is that it is the Meeowmix song…

Meeow-Mix song

Thankfully, an increasing number are not doing that at all: a glance through the range of MOOCs currently on offer via the (excellent) MOOC aggregator at http://www.class-central.com/ shows a range of lengths between 2 and 15 weeks as well as a goodly range of self-paced courses of somewhat indeterminate length. After early attempts mostly replicated university courses, the norm now appears to be around 6 weeks, and falling fast. The rough graphs below (that I created based on class-central’s data) of those starting soon and those that have already finished illustrate this trend quite nicely. Note in particular the relative drop in 10-week and higher courses and the rise in those of 4, 6 and 8 weeks. While it is far from all being down to better teaching – some of the rise in shorter courses is notably due to a trend towards samplers that are intended to draw people in to fee-paying courses – there is a pattern here. And, to counterbalance such forces, it should be remembered that a fair number of the longer courses have ambitions to reintegrate their students within their paid-for broken systems, so they are sometimes timetabled with learning as a secondary consideration and so retain their infeasible length.

MOOC lengths till now…

MOOC lengths (past)

 

 

Mooc lengths for courses about to start…

MOOC lengths (future)

 

Getting away from courses

Though the interest in MOOCs is fuelled and sustained by the fact they are free (though sadly, increasingly not as open as they were in the halcyon days of cMOOCs), popular and online, the really interesting thing about them is the attention they are drawing to what is wrong with the notion, form and above all the length of the course. This little thing is the real revolution. It radically changes the power dynamics. If people begin to disaggregate their courses, making them shorter and less teacher-controlled, they will put learners ever more in control of their own learning, giving them choices and the power to make those choices. Better still, it means that teachers are starting to create courses without unnecessary time constraints that are the size they need to be for the subject being taught. Pedagogy, though still not coming first, is playing a more significant role. But this is just a step in the right direction.

The power of small things

People who question completion rates for MOOCs almost never ask those same questions about Q&A sites, Wikipedia, Khan Academy, Fixya or How-Stuff-Works tutorials, OERS and Google Search. Indeed, the notion of ‘completion’ probably means nothing significant for such just-in-time tools: they are useful, or they are not, they work or they don’t. People use them or they don’t. You might waste a few minutes here and there on things that are unhelpful and those minutes add up but, on the whole, just-in-time learning does what it says on the box. And people use these tools because they need to learn. If someone needs to or wants to learn, you have to try really hard to stop them. But just-in-time is not always the way to go.

Clubs, not courses

I am not a great programmer but it is something I have been doing from time to time for about 30 years. When I’m stuck, I increasingly turn to StackOverflow, a brilliant set of sites based around a collectivized form of discussion forum – a bit more sophisticated than Reddit, a bit less intimidating that SlashDot (which remains perhaps the greatest of all learning tools for anyone with geek tendencies, but that needs a fair bit of skill and effort to get the most out of). StackOverflow doesn’t have courses, but it does have answers, it does have discussions, and it does have some very powerful tools for finding answers that are reliable, useful and appropriate to any particular need. The need can range from the very specific and esoteric (‘why am I getting this error?’) to matters of principle (‘what methodology is best for this problem?’) to general learning (‘what’s the best way to get started in Ruby-on-Rails?’) and everything in between. It’s like having your own immensely wise team of personal tutors, without a beginning date, an end date, or a fixed schedule of activities. This is not a course – it’s more like a Massive Open Online Club, with no restrictions to membership, no commitments, no threshold to joining. Conveniently, this has the same acronym as a MOOC. In fact, just as MOOCs subtly transform the social contract that is involved with traditional courses, so these ‘clubs’ are not exactly like their hierarchical, closed, membership-based forebears. They are what Terry Anderson and I have described as sets: not exactly a network of people you know, certainly not a hierarchically organized system like a group, just a bunch of people with a shared interest, some of whom know more than others about some things.

But what about accreditation?

Why should accreditation be something that happens only in and as a result of a course? It is bizarre and open to abuse that the people who teach a course should also be its accreditors. It is strange in the extreme that they should be the ones to say that students have ‘failed’ when it is obvious that this failure is not just on the part of the students but also of their teachers, which makes those teachers very poor and biased judges of success. It might be just about acceptable if those teachers really are the only ones who know the topic of the course but that is rare. In Eire, students have a right to write and defend a PhD (by definition a unique bit of learning) in Gaelic. Despite the fact that the number of Gaelic speakers who are also experts in many PhD topics is not likely to be huge (unless the topic is Irish history or somesuch) they still manage to find expert examiners for them. It can be done.

At Athabasca University we have a challenge for credit option for many of our courses that can be used to demonstrate competence for certification purposes. Alternatively, if the match in knowledge is not precisely tuned to the credentials we award, we and many others have PLAR or APEL processes that typically use some form of portfolio to demonstrate competence in an area. And then there are upcoming and increasingly significant trends like the move to Open Badges, closed LinkedIn endorsements, gamified learning, or good old fashioned h-index scores that sometimes tell us more, at least as reliably, and in some ways in greater detail than many of our traditional accreditation methods.

There is seldom a good reason to closely link accreditation and learning and every reason not to.  Giving rewards or punishments for learning is the academic equivalent of celery – to digest it consumes more calories than it actually provides, distorting motivation so much that it demotivates.

Summing up

I have no doubt that some people might bemoan the loss of attention implied by just-in-time learning or this weakly structured club-oriented perspective on learning which has no distinct beginning and no specific end. It is true that courses do sometimes include things like ‘problem solving’, ‘argument’, ‘enquiry’, ‘research’ and ‘creativity’ among their intended outcomes and, assuming they provide opportunities to exercise and develop such skills, that’s a lot better than not having them. And some (indeed, many) courses are a genuinely good idea, because it really does take x amount of time to learn some things (where x is a large number) and learning works much more smoothly when you learn with other people and have a specific goal in mind. But many are not such a good idea, and most get the value of x completely wrong. No more should we assume that a 10-week (or 100-hour) course is the right amount of time needed to learn something than we should assume that the answer to teaching is a one-hour lecture (even though it sometimes really is part of a good answer).

There are those who cynically believe that the sole purpose of going to a university is to build a network of contacts and gain credentials that will be valuable in a future career, so you can do what you like to students while they are in college and it won’t matter a bit. In fact, there’s a fair bit of research that shows that it typically doesn’t, which is yet another reason to express concern that we are not doing it right. If that were really what universities were about then I would stop teaching now because it would be boring and pointless. I think that, if we claim that what we are doing is teaching then we should at least try to do so. But accredited, fixed-length courses get in the way of doing that.

It is true that much of the really interesting learning that goes on in courses is not really about the topic, but the process of learning itself – that is why there is a vague and hard to pin down notion of graduateness that makes a fair bit of sense even if it cannot be well expressed or measured, a problem that Dave Cormier and others have grappled with in interesting ways. I’m not at all against lengthy learning paths if that is what is needed to learn, nor do I object at all to letting someone guide you along that path if that is what will get you where you want to be, and I am very much in favour of learning with other people. My problem is that the fixed-size course with fixed learning outcomes and tightly integrated accreditation is not the only way, is seldom the best way, and is often the worst way to do it. The biggest thing that MOOCs are doing, and the most disruptive, is visibly disaggregating the learning process from the unholy alliance of mediaeval bureaucracy and Victorian accreditation methods. As long as MOOCs retain the form and structure of courses that are tied to these unholies, they will (from their purveyors’ rather than their students’ perspectives) mostly fail, and that is a good thing. Even cMOOCs, that deliberately eschew learning outcomes and fixed accreditation, still often fall into a trap of fixed lengths and processes. If we can learn something from that then they have served a useful purpose.

So there you have it – another long, opinionated piece about MOOCs with little empirical data and a lot of hot air. But I think the central point, that fixed course lengths and integrated accreditation lie at the heart of much that is wrong with traditional university education and that MOOCs bring that absurdity into sharp relief, is worth making. I hope you agree.

Afterword

You may have seen my recent post on MOOPhDs and might be wondering whether I am contradicting myself here. Well, maybe a little, and there was a little hint of satirical intent when I first suggested the idea that attempted to exaggerate the concept of the MOOC to show the absurdity of courses. But the MOOPhD idea grew on me and it actually makes a little sense – it does not demand fixed length courses and completely separates out the accreditation from the process, and is far more like an open club or support network than an open course. Indeed, the way PhDs, at least those that follow a vaguely European model, tend to be taught provides an expensive-to-implement but workable model of learning that entirely (or, following a sad trend towards great bureaucratization in some countries, to a moderate extent) avoids courses. So, universities do know how to break the chains. Most just haven’t yet figured out how to do that for their mass-produced courses.

MOOCs are so unambitious: introducing the MOOPhD

Massive Open Online PhDs

During my recent visit to Curtin University, Torsten Reiners, Lincoln Wood and I started brainstorming what we think might be an interesting idea. In brief, it is to build and design what should eventually become a massive, open, online PhD program. Well, nearly. This is a work in progress, but we thought it might be worth sharing the idea to help spark other ideas, get feedback and maybe gather a few people around us who might be interested in it.
The starting point for this was thinking about ways of arranging crowd funding for PhD students, which evolved into thinking about other crowd-based/funded research support tools and systems to support that. For example, we looked at possible ways to not only crowd-fund research projects but to provide structures and tools to assist the process: forming and setting up project teams, connecting with others, providing project management support, proposal writing assistance, presenting and sharing results, helping with the process of writing reports and papers for publication, and so on. Before long, what we were designing began to look a little like a research program. And hence, the MOOPhD (or MOOD – massive open online doctorate).
A MOOPhD is a somewhat different kind of animal from a MOOC. It is much longer and much bigger, for a start – more of a program than a course. For many students it might, amongst other things, encapsulate a variety of MOOCs that would help them to gain knowledge of the research process, including a range of research methods courses and perhaps some more specific subject-related courses.  This is quite apart from the central process of supporting the conduct of original research that would form the ‘course’ itself.
A MOOPhD will also attract a very different kind of learner from those found in most MOOCs, notwithstanding the fact that, so far, a lot of MOOC-takers already have at least a first degree, not uncommonly in the same subject area as the MOOC.
Perhaps the biggest difference between a MOOPhD and a MOOC, at least of the xMOOC variety, is the inevitable lack of certainty about the path to the destination. MOOCs usually have a fairly fixed and clear trajectory, as well as moderately fixed content and coverage.  Even cMOOCs that largely lack specified resources, outcomes and assessments, have topics and timetables mapped out in advance. While the intended outcomes of a PhD are typically pretty clear (the ability to perform original and rigorous research, to write academically sound papers and reports, to design a methodology, review literature, etc), and there are commonalities in the process and landmarks along the way, the paths to reaching those goals are anything but determined. A PhD, to a far greater degree than most courses and lower level programs, specifies a method and processes, but not the content or pathways that will be taken along the way. This raises some very interesting and challenging questions about what we mean by ‘course’ and the wisdom and validity of MOOCs in general, but discussion of that can wait for another post. Suffice to say, it is a bit different from what we have seen so far.
There are many existing sites and systems that provide at least some of the tools and methods needed. I have had peripheral involvement with a support network for students investigating learning analytics, for example, and have helped to set up a site to provide resources for graduate students and their supervisors. There are commercial sites like academia.edu and ResearchGate that connect academics, including graduate students. There are some existing MOOCs on research methods and crowd-funding sites to help with fees and kick-starting projects such as http://www.rockethub.com/ or www.razoo.com.  And, of course, there is the complete system of journal and conference reviewing that provides invaluable feedback for nascent researchers. Like all technologies, what we are thinking about involves very little if anything that is radically new, but is mostly an assembly of existing pieces. 
It is likely that, for many, a PhD or other doctorate would not be the final outcome. People would pick and choose the parts that are of value, helping them to set up projects, write papers or form networks. Others might treat it as a useful resource for a more traditional doctoral learning journey.
 

So what might a MOOPhD look like? 

A MOOPhD would, of necessity, be highly modular, offering student-controlled support for all parts of the research process, from research process teaching, through initial proposals, through project management, through community support, through paper writing etc. Students would choose the parts that would be of value to them at different times. Different students would have different needs and interests, and would need different support at different points along the journey. For some, they might just need a bit of help with writing papers. For others, the need might be for gaining specific skills such as statistical analysis or learning how to do reviews.  More broadly, the role of a supervisory team in modelling practice and attitudes would be embedded throughout.
Importantly, apart from badges and certificates of ‘attendance’, a MOOPhD would not be concerned with accreditation. We would normally expect existing processes for PhDs by publication that are available at many institutions to provide the summary assessment, so the program itself would simply be preparation for that. As a result of this process, students would accrue a body of research publications that could be used as evidence of a sustained research journey, and a set of skills that would prepare them for viva voces and other more formal assessment methods. This would be good for universities as they would be able to award more PhDs without the immense resources that are normally needed, and good for students who would need to invest less money (and maybe be surrounded by a bigger learning community).
 

Some features and tools

A MOOPhD might contain (amongst other things):
  • A community of other research students, with opportunities to build and sustain networks of both peers (other students) and established researchers
  • MOOCs to help cover research methods, subject specialisms, etc
  • A great deal of scaffolding: resources to help explain the process, information about everything from ethics to citation, means and criteria to self-assess such as wizards, forms and questionnaires, guidelines for reviewing papers, etc
  • Mentors (not exactly supervisors – too tricky to deal with the numbers)  including both experienced academics and others further on in the PhD process. Mentors might provide input to a group/action learning set of students rather than to individuals, and thus allow students to observe behaviours that the academics model.
  • Exemplars – e.g. marked-up reviews of papers. This is vital as one of the ways of allowing established academics to provide role models and show what it means to be an academic
  • Plentiful resources and links relevant to the field (crowd-generated)
  • A filtering and search system to help identify people and things 
  • A means to provide peer review to others (akin to an online journal submission system)
  • A means to have one’s own ideas and papers reviewed by peers
  • Tutorial support – most likely a variant on action learning sets to support the process. This would cover the whole process from brainstorming, to literature review, to methodology design, to conduct and analysis of research, to evaluation etc. Ideally, each set would be facilitated by a professional academic or at least an experienced peer.
  • A professionally peer reviewed journal system, with experienced academic editorial committees and reviewers (who would only see papers already ranked highly in peer review), leading to publication
  • Support for gaining funding – including crowd funding – for the research, particularly with regard to projects needing resources not already available
  • Support for finding collaborators
  • Support for managing the process – both of the whole venture as well as specific projects
  • Non-academic support – counselling and advice
  • Tools and resources to find accreditors – this is not about providing qualifications but preparing students so that they can easily get them

Some issues

There are some complex and significant problems to solve before this becomes a reality, including:

Accreditation

The main idea behind this is to prepare students for a PhD by publication, not to award doctorates. It is essentially about managing a research learning process and helping students to publish results. However, sustaining motivation over a long period without the promise of accreditation might be an issue.

Access to resources

One of the biggest benefits of an institution for a PhD student is access to closed journals and libraries. While it is possible to pay for such access separately from a course, and a system would certainly contain links to ways of discovering open articles, this could be an obstacle. Of course, while we would not and could not condone the use of the community to share closed articles, it is hard to see how we could police such sharing. 

Ethics

Without an institutional backdrop, there would be no easy way to ensure ethical research. Resources could be provided, action learning sets could be used to discuss such concerns, and counselling might be available (perhaps at a price) to help ensure that a process would be followed that wouldn’t pose an obstacle to gaining accreditation, but it would be difficult to ensure an ethically sound process was followed. This is an area where different countries, regions and universities follow different procedures anyway, and there is only broad uniformity around the world, so some flexibility would be needed.

Governance

Beyond issues of ethics, there is a need to find solutions to disputes, grievances, allegations of cheating etc. This might be highly distributed and enabled through crowd-based processes. A similar issue relates to ‘approvals’ of research projects: there would probably need to be something akin to the typical review processes that determine whether a student’s progress and/or proposed path are sufficient. It is likely that action learning sets could play a big role in assisting this process.

Subject specificity

The skills (and resources) needed for different types of PhD can vary enormously – the skills and resources needed by a mathematician are worlds away from those needed by someone engaged in literary criticism, which are worlds away from those needed by a physicist, astronomer or biologist. It would probably be too big a task to cater for all, and some might be all but impossible (e.g. if they require access to large hadron colliders or telescopes, or are performing dangerous, large scale or simply complex experiments). To some extent this is not the huge problem it first appears to be. It is likely that most of those interested in pursuing this process would already be either working in a relevant field (and thus have resources to call upon) or already be enrolled in an academic program, which would reduce some of the problem, but the chances are that the most likely areas where this process could successfully be applied would be those requiring few resources beyond a good brain, commitment and a computer. There are opportunities for multiple instances of this process across multiple subject areas and disciplines. Given our interests and constraints, we would probably aim in the first instance for people interested in education, technology, business, or some combination of these. However, there is scope for a much broader diversity of systems, probably linked in some ways to gain the benefits of common shared resources and a larger community.

Cold start

As the point of this is to leverage the crowd, it will be of little value if there is not already a crowd involved. The availability of high-quality resources, links and MOOCs might be sufficient to provide an initial boost to draw people to the system, as would a team of interesting mentors and participants, but it would still take a while to pick up steam.

Trust

In some fields, students are already reluctant to share information about their research, so this might be especially tricky in an open PhD process. Building sufficient trust in action learning sets and across the broader community may be problematic. Already, the openness needed for many MOOCs poses a challenge for some, but this process would require more disclosure an an ongoing basis than normal. This might be the price to be paid for an otherwise free program. However, the anticipated high drop-out rate would make it difficult to sustain tight-knit research groups/action learning sets over a prolonged period, and we would probably need to think more about cooperative than collaborative processes, so this may be difficult to manage. 

Start-up costs and maintenance

This will not be a cheap system to build, though development might be staggered. Resources would be needed for building and maintaining the server(s), creating content, managing the editing process for the journal, and so on. Potential funding models include start-up grants, company sponsorship (the value to organizations of a process like this could be immense), crowd-funding, subscription, advertising/marketing, etc. Selling lists of participants bothers me, ethically, but a voluntary entry onto a register that might be passed on to interested companies for a fee might have high value. While we might not award doctorates, those who could stay the course would clearly be very desirable potential employees or research team members.

Encouraging academics to participate

Altruism and social capital can sustain a relatively brief open course, but this kind of process would (unless a different approach can be discovered) require long term commitment and engagement by professional academics. There may be ways to provide value to academics beyond the pleasure of contributing and learning from students. For instance, students may be expected/required to cite academics as co-authors where those academics have had some input into the process, whether in feedback along the way or in reviewing/completing papers they have written, or may be granted access to data collected by students. This would provide some incentive to academics to help ensure the quality of the research, and help students by seeing an experienced academic’s thinking processes in action.

Summary

This is a work in progress and there are some big obstacles in the way of making it a reality. We would welcome any ideas, suggestions or expressions of interest!