I recently did a session at the University of Brighton’s Learning and Teaching Conference on the trouble with modules – the name used for what are more commonly known as ‘courses’ in North America, ‘units’ in Australia and ‘papers’ in New Zealand. A couple of people who missed the session have asked for more details than what was shown in the slides that I posted from the session, so this post is a summary of some of the main points. It is mostly gleaned from my notes that accompanied the short presentation part, tidied up and slightly expanded on a bit for the blog.  I have not gone into much detail about what would happen if we did away with courses altogether, nor described the results of any of the reflective activities that were involved in the original session as I have no notes on those parts and not enough time to write them. It does contain a bunch of ideas and suggestions about how to overcome some of the innate weaknesses of courses though, that I hope will have some value to somebody. If anything is unclear or arguable, I’m very happy to follow up via the comments on this post!

Why (most) courses are a bad idea

The taught university course as we know it today started out as nothing more than the study of a (single) book, in schools in pre-university times and in the early days of universities, nearly a thousand years ago. The master or lecturer would read the book and, perhaps, comment on it and discuss it with students. This made a lot of sense. Books were very expensive and rare objects, and so were scholars. It was by far the most efficient way to make use of a rival good (the teacher and/or the book) to reach as many people as possible. Whether or not it was the best way to learn, without it there would be no learning about or from the book at all. These efficiencies remained significant for the next 900 years or so after universities were invented (first in Bologna and, later, Paris, Oxford and the slow-moving flood that followed over the next few centuries, right up to the recent trend in MOOCs). The course slowly evolved into more subject-specific areas that often drew from many books and, later, papers, and the printing press made books slightly less of a luxury, but the general principle, that knowledge was thinly disributed and the most efficient way to make it available was one-to-many transmission in a physical room, continued to make sense. As universities grew, it was equally sensible that processes and architectures were designed to make this still more efficient. Timetables were used to schedule these scarce resources, lecture theatres designed to reach as many ears and eyes as possible, desks invented to take notes, blackboards invented to provide a source for them, written exams invented to make assessments easier to mark (the first were in 1789) and libraries and classification systems invented to store and retrieve books and periodicals. And, of course, if students and teachers were not around, there was no point in scheduling classes, so courses naturally divided around the holidays of Christmas, Easter and during harvest time in the summer, when (perhaps – this is disputed) students were called back to work on farms. All of this made perfect sense and made the best use of limited means – perhaps the only means that could have worked at all. And this is what we have inherited, whether or not we observe Christian holidays, whether or not we have almost free access to a cornucopia of information on the web and mobile devices, whether or not we have sophisticated information systems that make scheduling and organization of resources more flexible, or tools to connect us with anyone, anywhere, any time around the world. Around it we have built innumerable structures – notions of course equivalence that are related to accreditation and assessment, replicability, resource allocations, pay structures, etc – that have become very deeply embedded, not just within universities but in society as a whole. Universities have become gatekeepers that filter students as they come in and warrant their competencies as they leave, not just to become academics but to work in many occupations. And the unit of measurement is based around the course. Courses are so deeply embedded that, when people attempt educational reform, they are seldom even noticed, let alone questioned. If people want to make things better in education, they normally explicitly mean ‘better courses’. Even open and distance universities like Athabasca, that dumped prerequisites, the schedule and traditional lecture/tutorial/seminar format, adhere to the broad pattern of course length (measured now in hours of study, like most of the rest of the world outside North America) fixed outcomes and assessments.  Likewise, companies unwisely create or purchase courses for their employees to go out and learn stuff, albeit usually with fewer institutional constraints on timing, accreditation and format.  But there is no pedagogical reason whatsover that it should be this way.

What this means

The trouble is that courses, at least as they have mostly evolved, are not pedagogically neutral technologies. This is pretty obvious to anyone who has ever created one. It is a completely insane idea that every subject can be taught in multiples of precisely the same period or requires the same amount of study as every other. Typically (varying from place to place but usually unvaryingly within a given institution) this means 10-15 weeks or some multiple of that, or 100-200 hours of student effort. Taught courses, as we know them in our institutions today, have objectives and/or outcomes, and assessments to match, which conspire to mean that the intent is that everyone learns exactly the same thing or skill, whether they already know it, don’t need to know it, or not. Courses therefore differentiate – you pass them or fail them. Maybe you pass or fail them well or badly. As an incidental peculiarity, the blame for failure to teach is transferred to the students – they fail, not their teachers. This has big implications for an individual’s sense of self worth and on their ability to seek employment, and it impacts society (and individuals who suffer this process) deeply. Another consequence of this is that, thanks to the need for economies of scale and/or fitting things into timeslots or with other courses that might be similar, typically everyone is taught the same way on a given course, and taught the same things, whether or not it suits their needs, prior knowledge, interests and aspirations. While the notion of teaching to learning styles is palpable nonsense, there is no doubt that people have very different needs and preferences from one another, so parts of every course will bore or confuse some of their students some or all of the time and nearly all will contain parts of little or no relevance to a learner’s needs. None of this makes any pedagogical sense whatsoever. Bloom’s two-sigma problem (based on the fact that there is roughly a two sigma difference between results for those taught in traditional classrooms and those taught one-to-one) is a difficult challenge to address because, quite apart from their innate peculiarities, these features of the typical pattern followed by courses lead to one extremely big and elephant-in-the-room: they are inherently demotivating. 

Courses and motivation

People love to and need to learn, constantly and voraciously. It’s in our nature. If someone wants and/or needs to learn something, you have to do something pretty substantial to prevent them from doing so. Enter the taught course.

The first way that courses stand in the way of learning is, at first glance, relatively innocuous. The fixed nature and form of the course combined with its length necessarily means that, for the vast majority of students, parts will be boring, parts will be irrelevant, and parts will be over-taxing. This means most students’ need for challenge at an attainable level will not be met, at least some of the time.  It means that course content, process, rules of conduct, expectations and methods are strongly determined by someone else, sapping control away. Self-determination theory, a powerful construct that has been validated countless times over several decades, makes it very clear that, unless people feel in control, are challenged with achievable goals and experience relatedness, they will not be intrinsically motivated, no matter what other factors motivate them. Though often supporting relatedness (connection to something or someone beyond yourself), taught courses are, by and large, structured to reduce two of those three vital factors. It is no surprise then that teachers have to find ways to get around the lack of motivation engendered by the course format. There are a few teachers, sadly, who positively relish the exercise of their power, who enjoy rewarding and punishing students, who like to apply rigid control over behaviour in the classroom, who take a kind of sick pleasure in watching students suffer, who make students do things ‘because it’s for their own good’. They need our pity and support, but should not be allowed to teach until they have overcome this sickness. Luckily, by far the majority of us do our best to inspire, to actively encourage students to reflect on and actively align their intrinsic hopes and desires with what we are teaching, to offer flexibility and control, to empower students, to nurture their creativity, and to give some attention to each student. That’s the pleasure most of us get from teaching. We certainly don’t all succeed all of the time, even the best fail pretty regularly, and we could all improve, but at least we try. However, it’s an uphill battle.

This leads to the second and far more harmful effect of taught courses on motivation.  Most of us who work in higher education are constrained by the nature of the course and its accreditation to apply extrinsic rewards and punishments in the form of grades, even though we know it is a truly terrible idea. The reasoning behind the use of grades as motivators is understandable. We can easily observe that extrinsic methods do actually, on the whole, to some extent work, in the short term. Depending on the context, the effect can last from minutes to months. Indeed, behaviourists (who only ever did short-term studies) based a whole psychological movement on this idea. What is less obvious, and the most crucial structural disaster in the way the vast majority of courses are designed, is that they invariably and totally predictably utterly destroy any intrinsic motivation that people may already have, often irreparably. A big part of the reason for this is that it creates a locus of causality for a task or behaviour that is perceived as being controlled by someone or something else, so it does again come back to an issue of control, but this time the effects are devastating, not just reducing motivation but actively militating against it. This crowding effect has been demonstrated over and over again in well-designed and hard-to-refute research studies for decades. In many cases, rewards and punishments don’t even achieve what they set out to do in the first place. For example, companies that offer performance-related bonuses typically get lower performance from their workers, and daycare that punish parents who are late picking up their children find that parents actually pick them up even later. Worse, once the damage is done, it is very hard if not (sometimes) impossible to entirely undo it. It’s like the motivation pathways have been permanently short-circuited. Worse still, how we are taught is often a major factor in determining how we learn, and we come to expect and (like addicts) even depend on extrinsic motivation to drive us. This is one of the reasons I sometimes describe my role as ‘un-teaching’ – there is often a lifetime of awful learning habits to undo before we can even start. 

If you are not convinced, do check out a few of the hundreds of papers at http://www.selfdeterminationtheory.org/publications/ or read pretty much anything by Alfie Kohn, or Edward Deci, or Richard Ryan. There are plenty of studies from the field of education that look at the effects of rewards and punishments and find them worse than wanting.

Breaking the cycle

There are alternatives to typical institutional taught courses, some of them very common, others less so. The University of Brighton has a great program, the MSc/MA by Learning Objectives, in which students work with supervisors to develop a set of outcomes, a means of assessment, and a work plan to reach their goals. While there are a few time and process constraints here and there for practical reasons, they are not too onerous. Students on this program tend to pass it, not because its standards are low, but because everything is aligned with what they want and need to do. A few programs at Athabasca University have similarly flexible courses that act as a kind of catch-all to enable people to do things that matter to them.  PhD programs, of the traditional variety used in the UK, have (or had – the course-based American model is sadly becoming more prevalent) no obligatory courses and are entirely customized to and often by the individual student, with nothing but a few processes to ensure students remain on track and supported. They can take from 2-10 years to complete. This length can be a problem as our motivation usually changes over such a long time and extrinsic factors are often introduced that can affect it badly, but the general principle is a good one. Athabasca University’s challenge process makes it possible to completely separate accreditation from learning, which (almost) avoids the whole course problem altogether, though it does unfortunately only work if you happen to have the precise set of competences provided by actual taught courses. Its self-paced undergraduate courses, though still markedly constrained by a notional equivalence to their paced brethren, free students from the tyrrany of schedules, even if they do have other features that are overly limiting. PLAR/APEL processes that are common in institutions across the world separate learning from accreditation almost entirely. And that’s not to mention a huge host of teach-yourself methods and resources from Google Search to Wikipedia to the Khan Academy to Stack Exchange and hundreds of other fine online systems that most of us use when we actually want and need to learn something. And, of course, there are books, which have the great benefit of allowing us to skip things, re-read things, look up references and so on, so our paths through them are seldom linear and always under out control – unless we are forced to read them because of a course. 

But what about the run-of-the-mill?

Though there is much to be learned from existing methods that entirely or partially by-pass the harmful effects of taught courses, teachers in higher education operate under a set of ugly constraints that make it very difficult and often impossible for us to completely avoid their ill effects, especially when student numbers are large and things like professional standards bodies come into the picture. Until we achieve massive educational reform, which might allow us to provide multiple paths to achieving competence, that might separate learning from accreditation, that might be chunked in ways that suit the needs of learner and subject, we are mostly stuck with the offspring of a mediaeval system that has evolved to defend itself against change. Most of us have to grade things, we have  to make use of learning objectives/outcomes, and we don’t have much control over course length. Often, especially in lower-level courses and/or where standards bodies are involved, we have little control over the competences that need to be attained, whether or not we are competent to teach them. Moreover, many of the most effective existing methods of teaching without courses are very resource-hungry. It would be great to apply the (UK-style) PhD process to all of our teaching but it is economically infeasible. PhDs are expensive for a very good reason – many of the economic and physical constraints that drove the development of courses in the first place have not gone away, even though some have been notably diminished. Given these issues, I will finish this post with a few general ideas, suggestions and patterns to help reduce the ill effects of courses without destroying the system of which they are a part. 

Give control

Traditional teaching seems determined to take control away from learners, but we can do much to give it back. Amongst other things:

• allow students to choose what they do and how they do it. For instance, I have a web development course that centres around a site that students build throughout the course, that is about something they choose and they care about, and a course process that encourages them to choose between (or discover for themselves or their peers) multiple resources and methods to learn the requisite skills along the way. It makes extensive use of peer support and encourages sharing of problems and solutions, so that students teach one another as a natural fall-out of the process. It uses reflection to support the process, and an assessment based on evidence (that the students select for themselves) of meeting specified learning outcomes. It’s far from perfect, and it does often cause problems (especially at first) for those who have learned dependence via our broken educational system, but it shows one way that learners can take the reins.
• allow students to choose the learning outcomes. This is trickier to enact because of the rigid requirements we usually have to develop curricula and match them with those delivered elsewhere. However, if the outcomes we specify are not too specific, relating to broad competences, it is still possible to allow some flexibility to students to identify finer-grained outcomes that suit their needs and that are exemplars of the general overarching outcomes. I’ve found this approach easier to follow in graduate level courses in ill-defined subject areas – I don’t really have a way of doing this well for those that are constrained by disciplinary standards.
• allow students to design their own assessments. This one is easier. Learning contracts are one way to do this, supported with scaffolding that allows students to develop their own plans for assessment. Similarly, we can ask for them to provide evidence in a form that suits them (one of the best computing assignments I have ever seen was mostly done as poetry, and I once had a great explanation of the ISO model of network management explained using Santa Claus’s elves). At the very least, we can offer alternative pre-written forms of assessment that students can choose between according to their preferences.
• allow students to pick their own content. This is a trick I have used for several courses. I offer a menu of options that address the intended (broad) outcomes and negotiate which parts we/they will cover during the course. It takes a little more effort to prepare, but the payoff is large. For graduate level courses I sometimes encourage students to develop their own content that we all then use.
• allow students to choose their own tools, media, platforms, etc. Where possible, students should not be limited in their choice of technologies needed to complete the course. This can be tricky where we are constrained by things like institutional platforms, but there are often ways to allow at least some flexibility (e.g. mobile-friendly versions, PDF and e-book formats, standard formats that allow the use of any editor or development tool, etc)
• allow students to pick the time and place. This is the default at Athabasca University for most courses, but can be trickier when there are timetables and constraints of working with others according a schedule. Classroom flipping can help a bit, limiting what is done in the class to things that actually benefit from being somewhere with other people (feedback, dialogue, collaboration, problem-solving, etc), and leaving a lot to self-paced study. This is true online as well as in face-to-face teaching. Indeed, counter-intuitively, it is even one of the odd potential benefits of traditional lectures, inasmuch as they typically only take an hour of a student’s time once a week, between which students are free to learn as they please (not a completely serious point, but worth pointing out because of the important and universally applicable lesson it reminds us of, that teaching behaviours only have a tangential relationship with learning behaviours).
• allow students to control social interaction. I am a huge fan of learning with other people but we all have different needs for engagement with others in our learning, and it doesn’t suit everyone equally all the time. Where possible, I try to build processes that let those that benefit from social interaction to work with others, but that let those that prefer a different approach to work alone, using evidence-based assessments rather than process-based ones. For instance, evidence can include help given to others or conversations with others, but can as easily come from individual work (unless social competences are on the menu for learning). I find it useful to build simple sharing (as opposed to dialogue) into the process so that even the least sociable of students share things and therefore support the learning of others.

Use better forms of extrinsic motivation

Extrinsic motivation is not all equally awful and some is barely distinguishable or even a part of intrinsic motivation. Extrinsic motivators lie on a spectrum from bad (externally imposed reward and punishment) to much better and more internally regulated varieties, such as:

• doing things out of a sense of duty, guilt or obligation (introjected regulation) or, better,
• doing things because they are perceived as worthwhile in themselves (identified regulation, e.g. losing weight) or, better still,
• doing things because they are necessary steps to achieve something else we are really motivated to achieve (integrated regulation).

See http://www.selfdeterminationtheory.org/theory/ for more about these differentiations. There are plenty of ways to use this to our advantage. It can often, for instance, be useful to encourage reflection on a learning activity. This can be used to think about why we are doing something, how it relates to our needs and goals, and what it means to us. Reflection can kindle more effective forms of extrinsic motivation that are far less harmful than externally imposed rewards and punishments. It is also valuable to nurture community, so that students feel obligations to the team or to one another, and support one another when the going gets rougher. Also, seeing how others are motivated can inspire us to recognize similar motivations in ourselves. Shared reflections (e.g. via blogs) can be particularly valuable.

Grades are not always necessary. While getting rid of the need to summatively assess is seldom possible, we can often avoid the use of grades (pass/fail is a little better than a mark), and we can make it possible for students to keep at it without grading until it is right, thus reducing the chance of failure. My courses tend to have feedback opportunities scattered throughout but I explictly avoid giving any grades until the last possible moment. It can upset some students who have learned grade-dependence, so it is important that they are fully aware of the reasoning and intent, and that the feedback is good enough that they can judge for themselves how well they are doing (I don’t always get that bit right!). Of course, I am only suggesting that we lose the grades, not the useful feedback. Feedback is crucial to allowing students to feel in control – they need to know what they are doing well and what could be improved, and plentiful feedback can be hugely motivating, showing that other people care, contributing to a sense of achievement, and more. Good, descriptive feedback that focuses on the work (never the student) is a cornerstone of effective educational practice. Grades tell us little or nothing, while encouraging an extrinsic focus that is harmful to motivation.

Step outside the course

Making links beyond a single course can be very beneficial to motivation. I attended an interesting presentation (at the same conference this originated in) the other day by Norman Jackson who talks about lifewide as opposed to lifelong learning, an idea that captures this principle well. Creating opportunities for students to engage in external activities like (for example) clubs, societies, geological digs, competitions, community work, conferences, charitable work, kickstarters, Wikipedia articles, coding camps and so on can fill in a lot of motivational gaps, making it easier to see the relevance of a course, to feed new ideas into a course, to gain a greater sense of personal relevance and responsibility for one’s own learning, to expand on work done in a course in greater detail wihout the imposition of extrinsic motivation. Of course, students should be free to choose which of these they engage with and, better still, should find them for themselves. However, there is no harm in advertising such things, nor in designing courses that allow students to capitalize on learning from other activities within the course itself such as projects, show-and-tell sessions, flexible discussions and so on. There are also often opportunities for doing things across multiple courses, using outputs of one to feed another, or bringing together different skillsets for joint projects. Another way to reduce the harm slightly is to build multiple courses into a single overarching one, of lengths appropriate to the needs of the students and subject. 

Build learning communities and spaces rather than courses

Given the wealth of potential resources and people’s time that are available for free on the Web (not to mention in libraries) there is often no need to provide much, if any content (in the sense of stuff presenting subject matter). A couple of the most successful courses I have ever run have had no curriculum or content to speak of, just a set of broad outcomes, a very flexible and student-designed assessment, an approach to making use of the learning community and a responsive process to make it all happen. The process can take a surprising amount of time to develop, as it is important that it is both understood well by the students (including how it is assessed, expectations, norms, etc) and that it can be guaranteed to result in the intended outcomes (assuming these are not negotiated too). Getting that process and community right can be hard work both in the design phase and (especially) during the course but, when it does go right, it is very rewarding. I have often learned as much if not more than my students on those courses, and they are the only courses I have ever run with more than a couple of students where I have had nothing but grade A students (moderated by external examiners as well as by peers). The massive enthusiasm and passion that results from a rich learning community of learners who are in control of their own learning has to be seen to be believed. The essence of the method is to let go just enough but no more: a teacher’s role is to provide plentiful prodding, ideas, critical feedback and, above all, scaffolding so that students feel confident that they are making progress in useful directions (and get help when they are not). It is also a bit of a juggling act to make sure that even loose outcomes are met, especially as students tend to diverge in all sorts of different directions, some of which are brilliant and worth pursuing – getting those outcomes loose enough in the first place but sufficiently recognizable and relevant to academic careers is a bit of an art that I am still learning. It also takes a lot of energy and dedication to make it work so, if you are having a bad week or two, things can go topsy turvy pretty fast.  It is worth putting a huge amount of effort into the first few weeks, responding enthusiastically and personally at any time of day or night that you can afford in order to set the tone, show that you care, explain your approach and soothe any fears. Once you have established trust that you care, and have nurtured a strong learning community, students tend to help one another a lot and forgive you when you are less attentive later on. I try to design the process so that I can intentionally let go in later weeks too.

In conclusion

As an intrinsic design feature, traditional university taught courses and their attendant processes and regulations impose unnatural restrictions on both teachers and students, reducing control and stunting motivation. It would be great to throw off these restrictions altogether. We could make enormous gains simply through separating teaching from accreditation (at least, wherever possible – in extremely rare cases it really is true that there is only one person who can reliably judge competence and that person is the teacher). This may soon become a necessity rather than a virtue if MOOCs continue to evolve faster than the means to reliably accredit the results. Athabasca University already has the challenge process to cope with that, though is significantly fettered by the need to match competences achieved with those that apply to existing courses – our challenge process is insufficiently fine-grained to allow real flexibility. There would be equally great gains if we made courses the right size (typically though not necessarily small) to fit the needs of different students rather than shoehorning them to fit the needs of institutions. We have technologies than can take the hard work out of managing the ensuing complexity so traditional timetabling woes need not impede us, and it would make it much easier to mix and match, including to accredit learning done in different ways. However, there is plenty that can be done even within the constraints of a typical university course, as long as we are aware of the dangers and take steps to reduce the harm. I hope that this little piece and this smattering of suggestions has sparked an idea or two about how we might go about doing that. Perhaps, if more of us start to question the system and apply such ideas, it might help to make a climate where bigger change is possible. If you’re interested in finding out more, I have written about this kind of thing once or twice before, with slightly different emphases, such as at https://landing.athabascau.ca/blog/view/177831/the-monkeys-paw-effect-in-higher-education and at https://landing.athabascau.ca/blog/view/496760/cargo-cult-courses 

Thanks to Jim and several others who have recently brought my attention to IBM’s rather grandiose claim that, in a few years, classrooms will learn us. The kinds of technology described in this article are not really very new. They have been just around the corner since the 60s and have been around in quantity since the early 90s when adaptive hypermedia (AH) and intelligent tutoring systems (ITS) rose to prominence, spawning a great many systems, and copious research reported on in hundreds of conferences, books and journal articles. A fair bit of my early work in the late 90s was on applying such things to an open corpus, which is the kind of thing that has blossomed (albeit indirectly) into the recently popular learning analytics movement. Learning analytics systems are essentially very similar to AH systems but mostly leave the adaptation stage of the process up to the learner and/or teacher and tend to focus more on presenting information about the learning process in a useful way than on acting on the results. I’ve maintained more than a passing interest in this area but I remain a little on the edge of the field because my ambitions for such tools have never been to direct the learning process. For me, this has always been about helping people to help one another to learn, not to tell them or advise them on how to learn, because people are, at least till now, the best teachers and an often-wasted resource. This seemed intuitively obvious to me from the start and, as a design pattern, it has served me well. Of late, I have begun to understand better why it works, hence this post.

The general principle behind any adaptive system for learning is that there are learners, some kind of content, and some means of adapting the content to the learners. This implies some kind of learner model and a means of mapping that to the content, although I believe (some disagree) that the learner model can be disembodied in constituent pieces and can even happily exist outside the systems we build, in the heads of learners. Learning analytics systems are generally all about the learner model and not much else, while adaptive systems also need a content model and a means of bringing the two together.  

Beyond some dedicated closed-corpus systems, there are some big obstacles to building effective adaptive systems for learning, or that support the learning process by tracking what we are doing.  It’s not that these are bad ideas in principle – far from it. The problem is more to do with how they are automated and what they automate. Automation is a great idea when it works. If the tasks are very well defined and can be converted into algorithms that won’t need to be changed too much over time, then it can save a lot of effort and let us do things we could not do before, with greater efficiency. If we automate the wrong things, use the wrong data, or get the automation a little wrong, we create at least as many problems as we solve. Learning management systems are a simple case in point: they automated abstracted versions of existing teaching practice, thus making it more likely that existing practices would be continued in an online setting, even though they had in many cases emerged for pragmatic rather than pedagogic reasons that made little sense in an online environment. In fact, the very process of abstraction made this more likely to happen. Worse, we make it very much harder to back out when we automate, because we tend to harden a system, making it less flexible and less resilient. We set in stone what used to be flexible and open. It’s worse still if we centralize that, because then whole systems depend on what we have set in stone and you cannot implement big changes in any area without scrapping the whole thing. If the way we teach is wrong then it is crazy to try to automate it. Again, learning management systems show this in spades, as do many of the more popular xMOOC systems. They automate at least some of the wrong things (e.g. courses, grading, etc). So we had better be mighty sure about what we are automating and why we are doing it. And this is where things begin to look a bit worrying for IBM’s ‘vision’. At the heart of it is the assumption that classrooms, courses, grades and other paraphenalia of educational systems are all good ideas that are worth preserving. The problem here is that these evolved in an ecosystem that made them a sensible set of technologies at the time but that have very little to do with best practice or research into learning. This is not about learning – it is about propping up a poorly adapted system.

If we ignore the surrounding systems and start with a clean slate, then this should be a set of problems about learning. The first problem for learning analytics is to identify what are we should be analyzing, the second is to understand what the data mean and how to process them, the third to decide what to do about that. Our knowledge on all three stages is intermediate at best. There are issues concerning what to capture, what we can dicover about learners through the information we capture, and how we should use that knowledge to help them learn better. Central to all of this is what we actually know about education and what we have discovered works best – not just statistically or anecdotally, but for any and all individuals. Unfortunately, in education, the empirical knowledge we have to base this on is very weak indeed.

So far, the best we can come up with that is fairly generalizable (my favourite example being spaced learning) is typically only relevant to small and trivial learning tasks like memorization or simple skill acquisition. We’re pretty good at figuring out how to teach simple things well, and ITS and AH systems have done a pretty fair job under such circumstances, where goals (seldom learning goals – more often proxies like marks on tests or retention rates) are very clear and/or learning outcomes very simple. As soon as we aim for more complex learning tasks, the vast majority of studies of education are either specific, qualitative and anecdotal, or broad and statistical, or (more often than should be the case) both. Neither is of much value when trying to create an algorithmic teacher, which is the explicit goal of AH and ITS, and is implied in the teaching/learning support systems provided by learning analytics.  

There are many patterns that we do know a lot about, though they don’t help much here.  We know, for example, that one-to-one mastery teaching on average works really brilliantly – Bloom’s 2-sigma challenge still stands, about 30 years after it was first made. One-to-one teaching is not a process that can be replicated algorithmically: it is simply a configuration of people that allows the participants to adapt, interact and exchange or co-develop knowledge with each other more effectively than configurations where there is less direct contact between people.  It lets learners express confusion or enthusiasm as directly as possible, and for the teacher to provide tailored responses, giving full and undistracted attention. It allows teachers to directly care both for the subject and for the student, and to express that caring effectively. It allows targeted teaching to occur, however that teaching might be enacted. It is great for motivation because it ticks all the boxes on what makes us self-motivated. But it is not a process and tells us nothing at all about how best to teach nor how best to learn in any way that can be automated, save that people can, on the whole, be pretty good at both, at least on average.

We also know that social constructivist models can, on average, be effective, for probably related reasons. it can also be a complete disaster. But fans of such approaches wilfully ignore the rather obvious fact that lots of people often learn very well indeed without them – the throwaway ‘on average’ covers a massive range of differences between real people, teachers and learners, and between the same people at different times in different contexts. This shouldn’t come as a surprise because a lot of teaching leads to some learning and most teaching is neither one-to-one nor inspired by social constructivist thinking. Personally, I have learned phenomenal amounts, been inspired and discovered many things through pretty dreadful teaching technologies and processes, including books and lectures and even examined quizzes. Why does it work? Partly because how we are taught is not the same thing at all as how we learn. How you and I learn from the same book is probably completely different in myriad ways. Partly it is because it ain’t what you do to teach but how you do it that makes the biggest difference. We do not yet have an effective algorithmic way of making or even identifying creative and meaningful decisions about what will help people to learn best – it is something that people and only people do well. Teachers can follow an identical course design with identical subject matter and turn it into a pile of junk or a work of art, depending on how they do it, how enthusiastic they are about it, how much eye contact they make, how they phrase it, how they pace it, their intonation, whether they turn to the wall, whether they remembered to shave, whether they stammer etc, etc, etc, and the same differentiators may work sometimes and not work others, may work for some people sometimes and not others. Sometimes, even awful teaching can lead to great learning, if the learners are interested and learn despite rather than because of the teacher, taking things into their own hands because the teaching is so awful. Teaching and learning, beyond simple memory and training tasks, are arts and not sciences. True, some techniques appear to work more often than not (but not always), but there is always a lot of mysterious stuff that is not replicable from one context to the next, save in general patterns and paradigms that are mostly not easily reduced to algorithms. It is over-ambitious to think that we can automate in software something we do not understand well enough to turn into an algorithm. Sure, we learn tricks and techniques, just like any artist, and it is possible to learn to be a good teacher just as it is possible to learn to be a good sculptor, painter or designer. We can learn much of what doesn’t work, and methods for dealing with tricky situations, and even a few rules of thumb to help us to do it better and processes for learning from our mistakes. But, when it comes down to basics, it is a creative process that can be done well, badly or with inspiration, whether we follow rules of thumb or not, and it takes very little training to become proficient. Some of the best teachers I’ve ever known have used the worst techniques. I quite like the emphasis that Alexandra Cristea and others have put on designing good authoring environments for adaptive systems because they then become creative tools rather than ends in themselves, but a good authoring tool has, to date, proved elusive and far too few people are working on this problem.

‘Nothing is less productive than to make more efficient what should not be done at all’. Peter Drucker

The proponents of learning analytics reckon they have an answer to this problem, by simply providing more information, better aggregated and more easily analyzed. It is still a creative and responsive teacher doing the teaching and/or a learner doing learning, so none of the craft or art is lost,  but now they have more information, more complete, more timely, better presented, to help them with the task so that they can do it better. The trouble is that, if the information is about the wrong things, it will be worse than useless. We have very little idea what works in education from a process point of view so we do not know what to collect or how to represent it, unless all we are doing is relying on proxies that are based on an underlying model that we know with absolute certainty is at least partly incorrect or, at best, is massively incomplete. Unless we can get a clearer idea of how education works, we are inevitably going to be making a system that we know to be flawed to be more efficient than it was. Unfortunately, it is not entirely clear where the flaws lie especially as what may be a flaw for one may not be for another, and a flaw in one context may be a positive benefit in another.  When performing analytics or building adaptive systems of any kind, we focus on proxies like grades, attention, time-on-task, and so on – things that we unthinkingly value in the broken system and that mean different things to different people in different contexts.  Peter Drucker made an important observation about this kind of thing:

‘Nothing is less productive than to make more efficient what should not be done at all‘.

A lot of systems of this nature improve the efficiency of bad ideas. Maybe they valorize behaviourist learning models and/or mediaeval or industrial forms of teaching. Maybe they increase the focus on grading. Maybe they rely on task-focused criteria that ignore deeper connective discoveries. Maybe they contain an implied knowledge model that is based on experts’ views of a subject area, which does not normally equate to the best way to come by that knowledge. Maybe they assume that time on task matters or, just as bad, that less time spent learning means the system is working better (both and neither are true). Maybe they track progress through a system that, at its most basic level, is anti-educational. I have seen all these flaws and then some. The vast majority of tools are doing education-process analytics, not learning analytics. Even those systems that use a more open form of analytics which makes fewer assumptions about what should be measured, using data mining techniques to uncover hidden patterns, typically have risky systemic effects: they afford plentiful opportunities for filter bubbles, path dependencies, Matthew Effects and harmful feedback loops, for example. But there is a more fundamental difficulty for these systems.  Whenever you make a model it is, of necessity, a simplification, and the rules for simplification make a difference. Models are innately biased, but we need them, so the models have to be good. If we don’t know what it is that works in the first place then we cannot have any idea whether the patterns we pick out and use to help people guide their learning journeys are a cause, an effect or a by-product of something else entirely. If we lack an explicit and accurate or useful model in the first place, we could just again be making something more efficient that should never be done at all. This is not to suggest that we should abandon the effort, because it might be a step to finding a better model, but it does suggest we should treat all findings gathered this way with extreme scepticism and care, as steps towards a model rather than an end in themselves.

In conclusion, from a computing perspective, we don’t really know much about what to measure, we don’t have great grounds for deciding how to process what we have measured, and we don’t know much at all about how to respond to what we have processed. Real teachers and learners know this kind of thing and can make sense of the complexity because we don’t just rely on algorithms to think. Well, OK, that’s not necessarily entirely true, but the algorithms are likely at a neural network level as well as an abstract level and are probably combinatorially complex in ways we are not likely to understand for quite a while yet. It’s thus a little early to be predicting a new generation of education. But it’s a fascinating area to research that is full of opportunities to improve things, albeit with one important proviso: we should not be entrusting a significant amount of our learning to such systems just yet, at least not on a massive scale. If we do use them, it should be piecemeal and we should try diverse systems rather than centralizing or standardizing in ways that the likes of Knewton are trying to do. It’s bit like putting a computer in charge of decisions whether or not to launch nuclear missiles. If the computer were amazingly smart, reliable and bug-free, in a way that no existing computer even approaches, it might make sense. If not, if we do not understand all the processes and ramifications of decisions that have to be made along the way, including ways to avoid mistakes, accidents and errors, it might be better to wait. If we cannot wait, then using a lot of different systems and judging their different outputs carefully might be a decent compromise. Either way, adaptive teaching and learning systems are undoubtedly a great idea, but they are, have long been, and should remain on the fringes until we have a much clearer idea of what they are supposed to be doing. 

I’m preparing for a presentation and discussion tomorrow with some doctoral students on the orchestration of lifelong learning. Having come up with the topic some time ago on a whim I’m not entirely sure what I’ll be talking about, so this is mostly an attempt to focus my thinking a little and is very much a work in progress.

In brief, the central jumping off point for this discussion is that teachers are not isolated actors but are instead are gestalts formed from

• numerous technologies, including pedagogies, regulations, processes, techniques and tools,
• an uncountably large number of individuals and groups and, most notably of all,
• learners themselves.

For it to work, everything must harmonize or must make the right kinds of dischord to bring about learning. There are various things that shake out of this perpsective, not least of which being that there are many ways to organize this teaching gestalt that do not involve an educational system of the sort we are used to, and that do not involve individuals labelled as teachers. This matters because most of the learning we do throughout our lives does not take place in or result from formal education.

The teaching gestalt

Even and perhaps particularly in a traditional educational system, teachers are not just the ones that stand (metaphorically or actually) in front of classes and explicitly perform an act that we label as teaching. Teachers are also the authors, editors, illustrators, designers and publishers of textbooks, the builders of websites, the writers of articles and so on. Teachers are designers of school systems, timetablers, architects, designers and furniture builders. Teachers are makers of videos, programmers of online environments, system administrators, TV producers, designers of door handles and technicians. And, above all, learners are teachers – of themselves and of one another. In short, teaching is always a distributed role.

Unpicking this a little further, almost all learning transactions involve at least two teachers – the one with knowledge of content, process, etc, and the learner. Learning is always an active process of knowledge construction, linking, and sense-making in which we constantly reflect, reorientate, examine, and adjust our knowledge in the light of new information or new ways of seeing. We always teach ourselves at least as much as we are taught. We are not given knowledge – we make it. Another person may help to guide us, shape the directions we go, correct us when we are confused or wrong, and motivate us to go the extra mile, but we are always a teacher in this process, whether we like it or not.

In an educational context, a vast array of actors add their own contributions to the teaching whole. Some, like authors of textbooks, or creators of curricula, or other students sharing ideas and (mis)conceptions are very obviously playing a teaching role. Others are less obviously so, but they do matter. The people that made decisions about where to place a whiteboard, which tools to enable in an LMS, or what wattage of lightbulb to include in a classroom may make a huge contribution to the success of failure of a particular learning transaction. The designer of the timetable, the legislator who demanded a particular kind of content or a particular kind of behaviour, the setter of normalized tests, the curriculum designer and the person who cleaned the classroom, all play significant and sometimes crucial roles as part of the teaching gestalt. Timetables teach, LMSs teach, hallways teach. In an educational system it is the system that educates, not just the individual teacher. I particularly like the timetable example because it is a great rejoinder to those who rather naively suggest that teachers should put pedagogy first. Sure: but first you must do it only at these times, over this period, for this amount of time, in this physical or virtual place, on this subject. Whatever. Anyway, within this context, the person who is performing the explicit role of a teacher is thus just one of the teaching gestalt but, potentially, quite a special one, sometimes (but not always) second only to the learner in importance. He or she typically acts as a filter, conduit and interpreter that orchestrates this whole, that responds, gives feedback, shows caring. It’s not too surprising that we label this person differently from the rest of the gestalt.

Orchestral manoeuvring

Since we are talking about a process of orchestration, it is natural to think of music at this point, and the analogy works quite well. A teacher may be an orchestrator, adapting to a context in which many constraints and structures have already been determined by others, using the tools, techniques and technologies to play a part in the construction of knowledge that is hopefully the outcome. Some are conductors, trying to elicit harmonious learning through tight control of the process. Like the best conductors, the best teachers of this sort make use of the materials they are working with, fitting the strengths and weaknesses of the players, the acoustics of the venue, the nature of the instruments, to the demands of the piece to be played and the intended audience. Other teachers are more like arrangers, who organize the pieces and leave the playing to someone else. Some are like players in a band, maybe drummers or bassists providing a rhythm to keep learners on track, or perhaps as soloists showing virtuosity and improvisational skills that inspire the learners to new heights. Some are content to play second fiddle, bringing out the best in the soloist but always in the background. And then there are the ones who sit in a recording studio who play all the instruments themselves, sometimes even making the instruments, and arrange everything the way they want it to be arranged. Some play blues, using the same three chords and often simple technique to play an infinite and subtle range of tunes. Some play classically, sticking closely to but always interpreting a score. Some are composers. Some are jazz improvisors, modern or trad. Some go for unusual scales, exotic rhythms and peculiar blends, others prefer the folk traditions that they learned as children. The sounds that musicians make are a function of many things, including most notably the instrument itself as well as the surroundings in which it is played and the reactions of an audience. And, in most cases, there are many instruments to consider. A lot of the process of teaching is about the technologies tools and techniques, incredibly diverse, all of which have to work to a common purpose.

But whatever the tools, genres, blends and roles that teachers play, when it comes down to basics, teachers (that is to say, the players in the teaching gestalt) have to be skilled and creative, whatever and however they try to play. Above all, teaching (emerging from all the many contributors to that role) is a broad set of human practices, not a science, not just a set of techniques. It is, moreover, a creative, active and inventive practice that cannot be emptied of soul and programmed into a machine without losing the vitality and expression that makes it wonderful. This is not to suggest that machines cannot or should not be a big part of the process, however, any more than that an orchestra should try to play without instruments or a venue. Putting aside more blatant technologies like classrooms and LMSs, for better or worse, our educational systems are machines that, depending on your perspective and the aspect you are looking at, either enable or disable our ability to learn. Likewise, Google Search and Wikipedia (my two favourite e-learning technologies) have a very large and conspicuous machine element. And, of course, the creativity and inspiration can be distributed too. A bad teacher can be saved by a good textbook, for instance, and vice versa.

Why bother with teachers anyway?

It is tempting to say that most of the intentional learning we do is self-guided – that we teach ourselves anything from cooking to philosophy. I know it’s tempting, because I’ve been known to say it, and have read many research studies purporting to show this. However, this is nearly always massively wrong. What we actually do, in almost all cases, is to orchestrate teaching done by others. In some cases this is blatant and obvious. If we learn something by reading a Wikipedia article, or a book, or by watching a video, this is very clearly not a case of us teaching ourselves. At least, not totally. We are merely picking our teachers and exercising a bit of control over the pace, time and place that they teach us. We don’t get all the benefits of teaching that way by any means – importantly, we seldom get much in the way of feedback, for example, and any tailoring that happens is up to us. These kinds of things do not show us that they care about us. Such things are co-teachers, part of the teaching gestalt. But it is all a matter of degree: we are always our own teachers to some extent, and there are almost always others involved in teaching us, no matter how informal or formal the setting. Even when we learn by dabbling and experimenting, we are not exactly pure autodidacts. Partly this is because we often have some kind of target to aspire to because we have seen, read, heard or otherwise encountered terminal behaviours of the sort we are aiming for. For many competences, it is because the things we try to learn or learn with are typically designed by humans who have other humans in mind when they design them – this is true of learning that makes use of things like pencils, paints, cookware, computers, cars, musical instruments, exercise machines, calculators and yachts.  Learning in a vacuum is not possible, unless we are learning about the vacuum which might be, incidentally, one of those rare occasions where no other teacher is directly involved in the process.

By way of example, in recent years,  I have been ‘teaching myself’ to play a new instrument at least once a year. I know what these instruments sound like when they are played well, so I can recognize the gaps between what I can do with them and what they can do. Many teachers have taught me. I have seen other people playing them so I have a fair idea how to hold them but, on the whole, they are designed to be held and manipulated so it seldom takes too long to figure that out by trial and error. Their designers have taught me. That said, I challenge anyone to watch someone else play the flute and, based on what you get out of that, to make the flute sound the same. It’s mighty hard. You might get the odd note and you might even figure out how to shape your mouth differently to switch octaves, but simply copying is probably not quite enough. Most instruments have quirks like that and it would not normally be very wise to simply rely on trial and error. The actual process I generally follow usually involves reading a bit about fingerings, tunings, breathing, embouchure and so on, usually with instrument in hand so that I can check what it all means, then a lot of trial and error, lots of YouTube videos and a great deal of practice until I reach a plateau, after which the cycle repeats again as I learn how to do more advanced stuff like overtones, harmonics, complex chords, intonation, picking or bowing styles, etc. I am never going to become a virtuoso this way, sure, but it is loosely structured in a way that leads to a bit more than the outcome of a chopsticks culture (this refers to Alan Kay’s delightful analogy of what happens when you simply put a computer in a classroom and hope for the best). Eventually I need to play with other people who play better or differently, to get a bit of coaching, to find others who will challenge me to go beyond my comfort zone, but I generally wind up being competent to carry a tune reasonably enough before getting to that point. Part of the reason that I can do this kind of thing because I have learned to teach myself and, of course, I am building on a foundation of existing knowledge. I can read music. I’ve grappled with most families of musical instrument at some point. I know the difference between 3/4 and 4/4 time, and a little bit about harmony. And I know a little about how people learn. All of this is because I have had many teachers, very few of whom were intentionally playing that role.

The unsaid

This all leads to what will, in my talk tomorrow, be the jumping off point for the real discussion, and some questions to which I have some answers but mostly not the best ones. What do all the things that go up to make teachers actually do?  What is the value professional teachers add? How can we manage our teachers? How can we replace them? As professional teachers, how can we allow our students to manage us? What aspects of educational systems teach? What alternative ways of organizing and orchestrating learning might we discover, invent or adapt? I’m particularly interested in exploring ways to overcome some of the manifestly awful teaching that our educational systems do to our students like grading, for instance, and what to do when the tunes we want to play are not in harmony with those played by the systems we are working in. But I am also interested in exploring ways that we can enable people to be better orchestrators of their own inner and outer teachers, beyond institutional contexts, beyond xMOOCs, beyond simple tutorials. I’m hoping it will be a fun discussion. How best to characterize what I’m aiming for? A bit of jazz improvisation, perhaps.