Time to change education again: let's not make the same mistakes this time round

Posted , updated on

We might as well start with exams

In case anyone missed it, one of countless examples of mass cheating in exams is being reported quite widely, such as at http://www.ctvnews.ca/world/hundreds-expelled-in-india-for-cheating-on-pressure-packed-exams-1.2289032.

The videos are stunning (Chrome and Firefox users – look for the little shield or similar icon somewhere in or near you browser’s address field to unblock the video. IE users will probably have a bar appearing in the browser asking if you want to trust the site – you do. Opera, Konqueror and Safari users should be able to see the video right away), e.g.:

As my regular readers will know, my opinions of traditional sit-down, invigilated, written exams could not be much lower. Sitting in a high-stress environment, unable to communicate with anyone else, unable to refer to books or the Internet, with enormous pressure to perform in a fixed period to do someone else’s bidding, in an atmosphere of intense powerlessness, typically using a technology you rarely encounter anywhere else (pencil and paper), knowing your whole future depends on what you do in the next 3 hours, is a relatively unusual situation to find yourself in outside an exam hall. It is fair enough for some skills – journalism, for example, very occasionally leaves you in similar conditions. But, if it actually is an authentic skill needed for a particular field, then it should be explicitly taught and, if we are serious about it, it should probably be examined under truly authentic conditions (e.g. for a journalist, in a hotel room, cafe, press room, or trench). This is seldom done. It is not surprising, therefore, that exams are an extremely poor indicator of competence and an even worse indicator of teaching effectiveness. By and large, they assess things that we do not teach.

If that were all, I might not be so upset with the idea – it would just be weird and ineffective. However, exams are not just inefficient in a system designed to teach, they are positively antagonistic to learning. This is an incredibly wasteful tragedy of the highest order. Among the most notable of the many ways that they oppose teaching are that:

  • they shift the locus of control from the learner to the examiner
  • they shift the focus of attention from the activity to the accreditation
  • they typically punish cooperation and collaboration
  • they typically focus on content rather than performance
  • they typically reward conformity and punish creativity
  • they make punishments or rewards the reasons for performing, rather than the love of the subject
  • they are unfair – they reward exam skills more than subject skills.

In short, the vast majority of unseen written exams are deeply demotivating (naysayers, see footnote), distract attention away from learning, and fail to discriminate effectively or fairly. They make the whole process of learning inefficient, not just in the wasted time and energy involved surrounding the examination itself, but in (at the very least) doubling the teaching effort needed just to overcome their ill effects. Moreover, especially in the sciences and technologies, they have a strong tendency to reinforce and encourage ridiculous content-oriented ways of teaching that map some abstract notion of what a subject is concerned with to exercises that relate to that abstract model, rather than to applied practices, problem solving and creative synthesis – i.e. the things that really matter.  The shortest path for an exam-oriented course is usually bad teaching and it takes real creativity and a strong act of will to do otherwise. Professional bodies are at least partly culpable for such atrocities.

There is one and only one justification for 99% of unseen written exams that makes any sense at all, which is that it allows us to relatively easily and with some degree of assurance (if very expensively, especially given the harmful effects on learning) determine that the learner receiving accreditation is the one that has learned. It’s not the only way, but it is one of them. That sounds reasonable enough. However, as examples like this show in very sharp relief, exams are not particularly good at that either. If you create a technology that has a single purpose of preventing cheating, then cheats (bearing in mind that the only thing we have deliberately and single-mindedly taught them from start to finish is that the single purpose of everything they do is to pass an exam) will simply find better ways to cheat – and they do so, in spades. There is a whole industry dedicated to helping people to cheat in exams, and it evolves at least as fast as the technologies that we use to prevent it. At least twenty percent of students in North America admit to having at some point in the last year cheated in exams. Some studies show much higher rates overall – 58% of high school students in Canada, for example.  It is hard to think of a more damning indictment of a broken system than this. The problem is likely even worse in other regions of the world. For instance, Davis et al (2009) reckon a whopping 83% of Chinese and 70% of Russian schoolkids cheat on exams. Let me repeat that: only 17% of Chinese people claim never to have cheated in an exam. See a previous post of mine for some intriguing examples of how that happens. When something that most people believe to be wrong is so deeply endemic, it is time to rethink the whole thing. No amount of patching over and tweaking at the edges is going to fix this.

But it’s not just exams

This is part of a much broader problem, and it is a really simple and obvious one: if you teach people that accreditation rather than learning is the purpose of education, especially if such accreditation makes a massive difference to what kind and quality of life they might have as a result of having or not having it, then it is perfectly reasonable that they should find better ways of achieving accreditation, rather than better ways of learning. Even most of our ‘best’ students, the ones that put in some of the hardest work, tend to be focused on the grades first and foremost, because that is our implicit and/or explicit subtext. To my shame, I’m as guilty as anyone of having used grades to coerce: I have been known to annoy my students with a little song that includes the lines ‘If a good mark is what you seek, blog, blog, blog, every week’.  Even if we assume that student will not cheat (and, on the whole, mature students like those that predominate at Athabasca U do not cheat, putting the lie to the nonsense some have tried to promote about distance education leading to more cheating) it challenges teachers to come up with ways of constructively aligning assessment and learning, so that assessment actually contributes to rather than detracts from learning. With skill and ingenuity, it can be done, but it is hard work and an uphill struggle. We really shouldn’t have to be doing that in the first place because learning is something that all humans do naturally and extremely willingly when not pressured to do so. We don’t need to be forced to do what we love to do. We love the challenge, the social value, the control it brings. In fact, forcing us to do things that we love always takes away some or all of the love we feel for them. That’s really sad. Educational systems make the rods that beat themselves.

Moving forwards a little

We can start with the simple things first. I think that there are ways to make exams much less harmful. My friend and colleague Richard Huntrods, for example, simply asks students to reflect about what they have done on his (open, flexible and learner-centred) course. The students know exactly what they will be asked to do in advance, so there is no fear of the unknown, and there is no need for frantic revising because, if they have done the work, they can be quite assured of knowing everything they need to know already. It is a bit odd not to be able to talk with others or refer to notes or the Web, but that’s about all that is inauthentic. This is a low-stress approach that demands nothing more than coming to an exam centre and writing about what they have done, which is an activity that actually contributes substantially to effective learning rather than detracting from it. It is constructively aligned in a quite exemplary way and would be part of any effective learning process anyway, albeit not at an exam centre.  It is still expensive, it still creates a bit more stress for students who have learned to fear exams, but it makes sense if we feel we don’t know our students well enough or we do not trust them enough to credit them for the work they have done. Of course, it demands a problem- or enquiry-based, student-centred pedagogy in the first place. This would not be effective for a textbook wraparound or other content-centric course. But then, we should not be writing those anyway as little is more certain to discourage a love of learning, a love of the subject, or a satisfying learning experience. 

There are plenty of exam-like things that can make sense, in the right kind of context, when approached with care: laboratory exercises, driving tests, and other experiences that closely resemble those of the practice being examined, for example, are quite sensible approaches to accreditation that are aligned with and can even be supportive of the learning process. There are also ways of doing exams that can markedly reduce the problems associated with them, such as allowing conversation and the use of the Internet, open-book papers that allow students to come and go as needed, questions that challenge students to creatively solve problems, exams that use questions created by the students themselves, oral exams that allow examiners to have a useful learning dialogue with examinees, and so on. There are different shades of grey and not all are as awful as the worst, by any means. There are other ways that tend to work better – for instance, badges, portfolios, and many other approaches that allow us to demonstrate competence rather than compliance, that rely on us coming to know our students, and that allow multiple approaches and different skills to be celebrated – but not all exam-like things are as bad as the worst of them.

And, of course, if we avoid exams altogether then we can do much more useful things, like involving students in creating the assignments; giving feedback instead of grades for work done; making the work relevant to student needs, allowing multiple paths, different evidence; giving badges for achievement, not to goad it, etc, etc. There’s a book or two in what we can do to limit the problems though, ultimately, this can only take us so far because, looming at the end of every learning path at an institution, is the accreditation. And therein lies the rub.

Moving forwards a lot

The central problem that we have to solve is not so much the exam itself as the unbreakable linkage of teaching and accreditation. Exams are just a symptom of a flawed system taken to its obvious and most absurd conclusion. But all forms of accreditation that become the purpose of learning are carts driving horses. horse pulling car I recognize and celebrate the value of authentic and meaningful accreditation, but there is no reason whatsoever that learning and accreditation should be two parts of the same system, let alone of the same process.  It it were entirely clear that the purpose of taking a course (or any other learning activity – courses are another demon we need to think carefully about) were to learn, rather than to succeed in a test, then education would work a great deal better. We would actually be able to do things that support learning, rather than that support credit scores; to give feedback that leads to improvement, rather than as a form of punishment or reward; to allow students to expand and explore pathways that diverge rather than converge; to get away from our needs and to concentrate on those of our students; to support people’s growth rather than to stunt it by setting false goals; to valorize creativity and ingenuity; to allow people to gain the skills they actually need rather than those we choose to teach; to empower them, rather than to become petty despots ourselves. And, in an entirely separate process of assessment that teachers may have little or anything to do with at all, we could enable multiple ways to demonstrate learning that are entirely dissociated from the process. Students might use evidence from learning activities we help them with as something to prove their competence, but our teaching would not be focused on that proof. It’s a crucial distinction that makes all the difference in the world.  This is not a revolutionary idea about credentialling – it’s exactly what many of the more successful and enlightened companies already do when hiring or promoting people: they look at the whole picture presented, take evidence from multiple sources, look at the things that matter in the context of application, and treat each individual as a human being with unique strengths, skills and weaknesses, given the evidence available. Credentials from institutions may be part of that right now, but there is no reason for that idea to persist and plenty of alternative ways of showing skills and knowledge that are becoming increasingly popular and significant, from social network recommendations to open badges to portfolios. In fact, we even have pockets of such processes well entrenched within universities. Traditional British PhDs, for example, while they are examined through the thesis and an oral exam (a challenging but flexible process), are examined on evidence that is completely unique to the individual student. Students may target the final assessment a bit, but the teaching itself is not much focused on that. Instead, it is on helping them to do what they want to do. And, of course, there are no grades involved at all – only feedback.

Conclusion

It’s going to be a long slow struggle to change the whole of the educational system across most of the world, especially as there’s a good portion of the world that would be delighted to have these kinds of problems in the first place. We need education before we can have cheating. But we do need to change this, and exams are a good place to start. It changed once before, with far less research to support the change, and far weaker technologies and communication to enable it. And it changed recently. In the grand scheme of things, the first ever university exam of the kind we now recognize as almost universal was the blink of an eye ago. The first ever written exam of the kind we use now (not counting a separate branch for the Chinese Civil Service that began a millenium before) was at the end of the 18th Century (the Cambridge Tripos) and it was only near the end of the 19th Century that written exams began to gain a serious foothold. This was within the lifetime of my grandparents. This is not a tradition steeped in history – it’s an invention that appeared long after the steam engine and only became significant as the internal combustion engine was born.  I just hope institutions like ours are not heading back down the tunnel or standing still, because those heading into the light are going to succeed while those that stay in the shadows will at best become the laughing stock of the world. 

On the subject of which, do watch the video. It is kind-of funny in a way, but the humour is very dark and deeply tragic. The absurdity makes me want to laugh but the reality of how this crazy system is wrecking people’s lives makes me want to cry. On balance, I am much more saddened and angered by it than amused. These are not bad people: this is a bad system. 

Reference

Davis, S., Drinan, P., and Gallant, T. (2009). Cheating in School: What We Know and What We Can Do. West Sussex, UK: Wiley-Blackwell.

Footnote

I know some people will want to respond that the threat or reward of assessment is somehow motivating. If you are one of those, this postscript is for you. 

I understand what you are saying. That is what many of us were taught to believe and it is one way we justify persisting despite the evidence that it doesn’t work very well. I agree that it is motivating, after a fashion, very much like paying someone to do something you want them to do, or hitting them if they don’t. Very much indeed. You can create an association between a reward/punishment and some other activity that you want your subject to perform and, as long as that association persists, you might actually make them do it. Personally speaking, I find that quite offensive, not to mention only mildly effective at achieving its own limited ends, but each to their own. But notice how you have replaced the interest in the activity with an interest in the reward and/or the desire to avoid punishment. Countless research studies from several fields have pretty conclusively shown that both reward and punishment are strongly antagonistic to intrinsic motivation and, in many cases, actually destroy it altogether. So, you can make someone do something by destroying their love of doing it – good job. But that doesn’t make a lot of sense to me, especially as what they have learned is presumably meant to be of ongoing value and interest, to help them in their lives. It is my belief that, if you want to teach effectively, you should never make people learn anything – you should support them in doing so if that is what they want to do. It is good to encourage and enthuse them so that they want to do it and can see the value – that’s a useful teacher role – but it’s a whole different ballgame altogether to coerce them. Alas, it is very hard to avoid it altogether until we change education, and that’s one good reason (I hope you agree) we need to do that.

For further information, you could do worse that to read pretty much anything by Alfie Kohn. If you are seeking a broader range of in-depth academic work, try the Self Determination Theory site.

Interview with Terry Anderson in The Voice Magazine – a Publication of the Athabasca University Students Union

Posted , updated on

The second part of an interview with Terry Anderson (with a link to the first part) talking about his approach to teaching, a bit about his research, a bit about some of his many contributions to AU, and a fair bit about the Landing and its value. Great stuff!

Terry voices his impatience with the slow uptake of the Landing, which is understandable. It is a bit frustrating to know that it can (and, for some, already does) radically improve the learning and working experience for staff and students alike, providing social presence, control, and a massive range of learning opportunities and activities that go far beyond anything possible in closed course containers or generic tools like mailing lists and project sites, but that we only see a ‘mere’ 7000+ users, 400+ groups, dozens of courses, as well as scores of posts and thousands of external visitors every day!  I have always taken the view that critical passion trumps critical mass every time. That is what keeps the Landing going and that is what will sustain it for the long haul. It’s a slow but steady burner that keeps getting better and more useful with every single post and comment, and the word is spreading: deeper, more sophisticated use is notably rising, diversity is increasing, and people are visiting longer and more frequently. For a system developed on a shoe-string, with very little central funding or support, it has done pretty well for itself for well over 5 years and I hope and expect it will continue to do so for the foreseeable future. And it does continue to evolve – you can expect to see ongoing improvements, not just tweaks but major gains in functionality, over the coming months, and we are working on a major upgrade to the underlying software framework at the same time that should enable even more improvements. With any luck we might even eventually get a bit more support from central pockets that might really let it fly but, even without much of that, the site grows in importance and value all the time.

Address of the bookmark: http://www.voicemagazine.org/articles/featuredisplay.php?ART=10321

Understanding transactional distance in web-based learning environments: An empirical study – Huang – 2015 – British Journal of Educational Technology – Wiley Online Library

Posted , updated on

This is a very important paper examining and verifying Moore’s theory of transactional distance. Sadly it is not open but those with AU accounts will be able to access it via AU Library.

The paper mostly confirms an inverse relationship between dialogue and transactional distance (more dialogue reduces the distance), and a direct relationship between structure and transactional distance (more structure increases the distance), that autonomy tends to slightly reduce the distance, and that a lack of both structure and dialogue is very bad indeed. But it also provides an interestingly nuanced view that appears to show high levels of both structure and dialogue are better still, as Moore himself predicts. I’m a bit sad that they used one of my earlier papers rather than my later book, Control & Constraint in E-Learning, because the findings strongly confirm my thesis in that book that modern forms of social media make it possible to have both high structure and high dialogue and that, consequently, transactional distance is lowest of all in such circumstances.  This paper appears to show that best of all worlds appears to occur when students use blogs, wikis, Twitter, etc, with notably lower transactional distance reported when compared with email/discussion forums.

There are also some interesting correlations between age and perceived transactional distance that the authors (I think rightly) put down to greater autonomy – older students tend to perceive lower transactional distance and tend to be more autonomous. There are also unexplained correlations with ethnicity that I am as puzzled about as the authors – this is intriguing and demands further investigation. I suspect cultural factors may explain this rather than ethnicity per se.

Though the studies are well conducted, I have some concerns with the measures and definitions used: I am particularly bothered by the split of interactions between people, content and interfaces. Structure here is defined by higher levels of learner-content and learner-interface interactions. I have always found ‘interface’ to be a very puzzling distinction as it is the interface that mediates both dialogue and learner-content interactions, so it is neutral to structure or dialogue – it ain’t what it is, it is how it is done that matters. It is precisely that neutrality that makes the value of social media so high because, out of dialogue, structure in the interface can emerge and/or the structure of the interface can help to shape dialogue. It is also interesting that transactional distance is (quite rightly) not just seen as a distance between instructor and student but between students and students: this is correct because students are one another’s teachers too. Again, this helps to explain the increased value of modern social media in reducing transactional distance because dialogue becomes content in the process, in ways that it does not in discussion forums and email.

Address of the bookmark: http://onlinelibrary.wiley.com/doi/10.1111/bjet.12263/abstract?systemMessage=Wiley+Online+Library+will+be+disrupted+on+21st+March+from+10%3A30+GMT+%2806%3A30+EDT%29+for+up+to+six+hours+for+essential+maintenance.++Apologies+for+the+inconvenience.&userIsAuthenticated=false&deniedAccessCustomisedMessage=

Ivy League For Free: What One Man Learned By Crashing Elite Colleges For 4 Years

Posted , updated on

A thought-provoking article on a man who gate-crashed some top tier US university courses as well as here in Canada. This was both politically and personally motivated. The man not only attended lectures and seminars but also got access to networks, direct interaction with professors, and received many of the benefits of a traditional university education bar the accreditation and support.

Attendance at lectures is rarely monitored and it has long been touted as one of the best free entertainment options available, if you choose your lectures wisely. I think I first recall reading about it in Playpower, a book on alternative culture from the early 70s. As the article suggests, it is generally pretty well accepted in many institutions and, as long as it doesn’t harm paying students (in this case I’d guess it was actually beneficial), it is hard to see why anyone would object. It might be a little trickier to get away with attending small tutorials or workshops involving restricted resources (e.g needing logins to university computers), and it would not always be easy to get the accompanying documentation and schedules, but this seems eminently do-able for many courses. I don’t think it scales well – most universities would probably start to institute controls if a large number of non-paying students started to join in, especially if they used up rival resources like handouts or materials, or hog tutor time in ways that harmed others. In some subjects it would not work at all. But it is quite an interesting perspective on openness. Face-to-face can be open too – in fact, it might be the default.

Makes me wonder about closed online courses and open MOOCs. Are we really as open as we claim? I think not so much.

Address of the bookmark: http://www.fastcompany.com/3043053/my-creative-life/ivy-league-free-what-one-man-learned-by-crashing-elite-colleges-for-4-years

Four Reasons to Worry About "Personalized Learning" – Alfie Kohn

Posted , updated on

Alfie Kohn is one of my favourite writers and this is a good example of why. Kohn’s basic concern with personalized learning is that  “Each of us can do what he likes as long as he ends up fundamentally similar to everyone else” but he drills down a lot further. These are his fundamental worries with how personalized learning is being conceived and implemented:

1. The tasks have been personalized for kids, not created by them.

2. Education is about the transmission of bits of information, not the construction of meaning.

3. The main objective is just to raise test scores.

4. It’s all about the tech.

I share Kohn’s concerns – I am appalled by a lot of the adaptive hypermedia, learning analytics and intelligent tutoring systems that I have seen, dating back around three decades or more but apparently now reaching critical mass in the personalization movement. He is absolutely right to be afraid of how these things are being used and implemented not just for kids but for adults. The trend towards measurability, uniformity and control of students is a heinous crime against this and future generations, made much worse by the institutionalized power abuse that comes with the package.

On the other hand, I think that it doesn’t have to be that way. First of all, it is important to consider the granularity: adaptive systems can have a lot of value when they are available in small pieces that give learners the power to choose them as tools to support their personal learning (and, as Kohn points out, social learning). I am also not averse, on the whole, to tools that suggest and advise but that do not dictate. But mainly, like all things technological, from painting to bridge construction, it ain’t what you do, it’s the way that you do it that matters. It’s a problem when the process of learning is hardened into a toolset that is dictated by a small hierarchy and not socially negotiated. It’s the higher levels of control that are the most worrisome technologies in this assembly: the tools just make it easier to exert that control. We should fight against the standardized tests, the teaching for grades, the manipulation of learners for political ends. But, when that toolset simply adds to what we already have, it can make the whole system more creative, flexible and controllable by learners. It’s the difference between what Franklin calls ‘holistic’ and ‘prescriptive’ technologies.

Address of the bookmark: http://www.alfiekohn.org/blogs/four-reasons-worry-personalized-learning

Laws of the academic jungle

Posted , updated on

My former VC, Sir David Watson, who died yesterday after a short illness, was a gentle, wise and caring man from whom I learned much and who supported me, guided me and challenged me in myriad ways. He was a remarkable man: a great educator, a fine musician and a gifted leader. I believe that he knew not just the names of all his 2500+ staff, but also enough about each and every one of them to sustain a conversation about their interests, achievements and friends. He was modest enough to deny this but, in 15 years of working with him, I never saw him falter and nor did anyone I ever spoke with about it. It was a phenomenon.

I share his nine Laws of Academic Life (AKA Laws of the Academic Jungle) in fond remembrance…

  • Academics grow in confidence the farther away they are from their true fields of expertise (what you really know about is provisional and ambiguous, what other people do is clear-cut and usually wrong)
  • You should never go to a school or department for anything that is in its title (which university consults its architecture department on the estate, or – heaven forbid – its business school on the budget?)
  • The first thing a committee member says is the exact opposite of what she means (“I’d like to agree with everything the vice-chancellor has just said, but…”; or “with respect”…; or even “briefly”)
  • Courtesy is a one-way street (social-academic language is full of hyperbole, and one result is the confusion of rudeness – or even cruelty – with forthrightness; however, if a manager responds in kind, it’s a federal case)
  • On email, nobody ever has the last word
  • Somebody always does it better elsewhere (because they are better supported)
  • Feedback counts only if I agree with it
  • The temptation to say “I told you so” is irresistible
  • Finally, there is never enough money, but there used to be.
 
Sad to have lost one of the great educational leaders.

Address of the bookmark: http://www.timeshighereducation.co.uk/news/laws-of-the-academic-jungle/408835.article

Datawind Aakash Android Tablets

Posted , updated on

The cheapest tablet in this range is CAD$43, for which you get a 7″ screen device with WiFi, Bluetooth and limited but extendible storage, capable of web browsing, email, Skype, word-processing and e-reading. Not well, for sure, nor with any kind of battery life to speak of, and with a low resolution screen with a viewing position rather than range of angles.

But it’s $43 (Canadian)!

That’s less than plenty of internet-capable radios, MP3 players, electronic picture frames, or even sophisticated alarm clocks, all of which it can comfortably replace and actually do a better job.  In fact, it’s less than a meal for two (with drinks) at my local pub. The others in the range don’t add much apart from a front-facing camera and very slow mobile data ($55), up to 3G phone and a slightly better screen for the top-of-the range UbiSlate3G7 for $90. Not too bad a price for an unlocked if totally enormous smartphone, though not the cheapest around.

The UbiSlates are Canadian, though the primary market for them is India, where they can be purchased for even less, and can come with $2/month mobile Internet (some US versions come with unlimited mobile web browsing for about US$100 a year). I think I might get one of these for the hell of it. 

 

Address of the bookmark: http://ubislate.ca/compare.php

Transactional Distance among Open University Students: How Does it Affect the Learning Process? : European Journal of Open, Distance and E-Learning

Posted , updated on

Interesting study looking into transactional distance between online learners at a Greek open university, with some great qualitative findings.

The findings are very revealing about the role and nature of dialogue in online learning at the authors’ university. As we noted in our book, Teaching Crowds, transactional distance becomes very complex once there are multiple ‘teachers’ (or teaching presences) involved, where peer and content interactions are multi-dimensional and so transactional distance shifts and varies all the time. The study reveals some quite nuanced and differentiated communication patterns that demonstrate this quite nicely.  A bit of fuzziness shows through, however, where what is reported is mainly levels of communication rather than perceived transactional distance. The two are very closely related, inasmuch as communication is a necessary but not sufficient condition for reducing transactional distance, but they are not the same thing. 

I find it hard to imagine, as suggested for future study in the conclusion, what ways one might measure transactional distance in learner-content or learner-interface interactions that would not make the distance extraordinarily high. This is almost true by definition, apart from in ‘creepy’ ways (e.g. if the learners felt psychological closeness and attachment with an AI) or, maybe stretching the definition a bit, through guided didactic conversation. I will be interested to see how the writers address this!

Address of the bookmark: http://www.degruyter.com/view/j/eurodl.2014.17.issue-1/eurodl-2014-0002/eurodl-2014-0002.xml