Time to change education again: let's not make the same mistakes this time round

Posted , updated on

We might as well start with exams

In case anyone missed it, one of countless examples of mass cheating in exams is being reported quite widely, such as at http://www.ctvnews.ca/world/hundreds-expelled-in-india-for-cheating-on-pressure-packed-exams-1.2289032.

The videos are stunning (Chrome and Firefox users – look for the little shield or similar icon somewhere in or near you browser’s address field to unblock the video. IE users will probably have a bar appearing in the browser asking if you want to trust the site – you do. Opera, Konqueror and Safari users should be able to see the video right away), e.g.:

As my regular readers will know, my opinions of traditional sit-down, invigilated, written exams could not be much lower. Sitting in a high-stress environment, unable to communicate with anyone else, unable to refer to books or the Internet, with enormous pressure to perform in a fixed period to do someone else’s bidding, in an atmosphere of intense powerlessness, typically using a technology you rarely encounter anywhere else (pencil and paper), knowing your whole future depends on what you do in the next 3 hours, is a relatively unusual situation to find yourself in outside an exam hall. It is fair enough for some skills – journalism, for example, very occasionally leaves you in similar conditions. But, if it actually is an authentic skill needed for a particular field, then it should be explicitly taught and, if we are serious about it, it should probably be examined under truly authentic conditions (e.g. for a journalist, in a hotel room, cafe, press room, or trench). This is seldom done. It is not surprising, therefore, that exams are an extremely poor indicator of competence and an even worse indicator of teaching effectiveness. By and large, they assess things that we do not teach.

If that were all, I might not be so upset with the idea – it would just be weird and ineffective. However, exams are not just inefficient in a system designed to teach, they are positively antagonistic to learning. This is an incredibly wasteful tragedy of the highest order. Among the most notable of the many ways that they oppose teaching are that:

  • they shift the locus of control from the learner to the examiner
  • they shift the focus of attention from the activity to the accreditation
  • they typically punish cooperation and collaboration
  • they typically focus on content rather than performance
  • they typically reward conformity and punish creativity
  • they make punishments or rewards the reasons for performing, rather than the love of the subject
  • they are unfair – they reward exam skills more than subject skills.

In short, the vast majority of unseen written exams are deeply demotivating (naysayers, see footnote), distract attention away from learning, and fail to discriminate effectively or fairly. They make the whole process of learning inefficient, not just in the wasted time and energy involved surrounding the examination itself, but in (at the very least) doubling the teaching effort needed just to overcome their ill effects. Moreover, especially in the sciences and technologies, they have a strong tendency to reinforce and encourage ridiculous content-oriented ways of teaching that map some abstract notion of what a subject is concerned with to exercises that relate to that abstract model, rather than to applied practices, problem solving and creative synthesis – i.e. the things that really matter.  The shortest path for an exam-oriented course is usually bad teaching and it takes real creativity and a strong act of will to do otherwise. Professional bodies are at least partly culpable for such atrocities.

There is one and only one justification for 99% of unseen written exams that makes any sense at all, which is that it allows us to relatively easily and with some degree of assurance (if very expensively, especially given the harmful effects on learning) determine that the learner receiving accreditation is the one that has learned. It’s not the only way, but it is one of them. That sounds reasonable enough. However, as examples like this show in very sharp relief, exams are not particularly good at that either. If you create a technology that has a single purpose of preventing cheating, then cheats (bearing in mind that the only thing we have deliberately and single-mindedly taught them from start to finish is that the single purpose of everything they do is to pass an exam) will simply find better ways to cheat – and they do so, in spades. There is a whole industry dedicated to helping people to cheat in exams, and it evolves at least as fast as the technologies that we use to prevent it. At least twenty percent of students in North America admit to having at some point in the last year cheated in exams. Some studies show much higher rates overall – 58% of high school students in Canada, for example.  It is hard to think of a more damning indictment of a broken system than this. The problem is likely even worse in other regions of the world. For instance, Davis et al (2009) reckon a whopping 83% of Chinese and 70% of Russian schoolkids cheat on exams. Let me repeat that: only 17% of Chinese people claim never to have cheated in an exam. See a previous post of mine for some intriguing examples of how that happens. When something that most people believe to be wrong is so deeply endemic, it is time to rethink the whole thing. No amount of patching over and tweaking at the edges is going to fix this.

But it’s not just exams

This is part of a much broader problem, and it is a really simple and obvious one: if you teach people that accreditation rather than learning is the purpose of education, especially if such accreditation makes a massive difference to what kind and quality of life they might have as a result of having or not having it, then it is perfectly reasonable that they should find better ways of achieving accreditation, rather than better ways of learning. Even most of our ‘best’ students, the ones that put in some of the hardest work, tend to be focused on the grades first and foremost, because that is our implicit and/or explicit subtext. To my shame, I’m as guilty as anyone of having used grades to coerce: I have been known to annoy my students with a little song that includes the lines ‘If a good mark is what you seek, blog, blog, blog, every week’.  Even if we assume that student will not cheat (and, on the whole, mature students like those that predominate at Athabasca U do not cheat, putting the lie to the nonsense some have tried to promote about distance education leading to more cheating) it challenges teachers to come up with ways of constructively aligning assessment and learning, so that assessment actually contributes to rather than detracts from learning. With skill and ingenuity, it can be done, but it is hard work and an uphill struggle. We really shouldn’t have to be doing that in the first place because learning is something that all humans do naturally and extremely willingly when not pressured to do so. We don’t need to be forced to do what we love to do. We love the challenge, the social value, the control it brings. In fact, forcing us to do things that we love always takes away some or all of the love we feel for them. That’s really sad. Educational systems make the rods that beat themselves.

Moving forwards a little

We can start with the simple things first. I think that there are ways to make exams much less harmful. My friend and colleague Richard Huntrods, for example, simply asks students to reflect about what they have done on his (open, flexible and learner-centred) course. The students know exactly what they will be asked to do in advance, so there is no fear of the unknown, and there is no need for frantic revising because, if they have done the work, they can be quite assured of knowing everything they need to know already. It is a bit odd not to be able to talk with others or refer to notes or the Web, but that’s about all that is inauthentic. This is a low-stress approach that demands nothing more than coming to an exam centre and writing about what they have done, which is an activity that actually contributes substantially to effective learning rather than detracting from it. It is constructively aligned in a quite exemplary way and would be part of any effective learning process anyway, albeit not at an exam centre.  It is still expensive, it still creates a bit more stress for students who have learned to fear exams, but it makes sense if we feel we don’t know our students well enough or we do not trust them enough to credit them for the work they have done. Of course, it demands a problem- or enquiry-based, student-centred pedagogy in the first place. This would not be effective for a textbook wraparound or other content-centric course. But then, we should not be writing those anyway as little is more certain to discourage a love of learning, a love of the subject, or a satisfying learning experience. 

There are plenty of exam-like things that can make sense, in the right kind of context, when approached with care: laboratory exercises, driving tests, and other experiences that closely resemble those of the practice being examined, for example, are quite sensible approaches to accreditation that are aligned with and can even be supportive of the learning process. There are also ways of doing exams that can markedly reduce the problems associated with them, such as allowing conversation and the use of the Internet, open-book papers that allow students to come and go as needed, questions that challenge students to creatively solve problems, exams that use questions created by the students themselves, oral exams that allow examiners to have a useful learning dialogue with examinees, and so on. There are different shades of grey and not all are as awful as the worst, by any means. There are other ways that tend to work better – for instance, badges, portfolios, and many other approaches that allow us to demonstrate competence rather than compliance, that rely on us coming to know our students, and that allow multiple approaches and different skills to be celebrated – but not all exam-like things are as bad as the worst of them.

And, of course, if we avoid exams altogether then we can do much more useful things, like involving students in creating the assignments; giving feedback instead of grades for work done; making the work relevant to student needs, allowing multiple paths, different evidence; giving badges for achievement, not to goad it, etc, etc. There’s a book or two in what we can do to limit the problems though, ultimately, this can only take us so far because, looming at the end of every learning path at an institution, is the accreditation. And therein lies the rub.

Moving forwards a lot

The central problem that we have to solve is not so much the exam itself as the unbreakable linkage of teaching and accreditation. Exams are just a symptom of a flawed system taken to its obvious and most absurd conclusion. But all forms of accreditation that become the purpose of learning are carts driving horses. horse pulling car I recognize and celebrate the value of authentic and meaningful accreditation, but there is no reason whatsoever that learning and accreditation should be two parts of the same system, let alone of the same process.  It it were entirely clear that the purpose of taking a course (or any other learning activity – courses are another demon we need to think carefully about) were to learn, rather than to succeed in a test, then education would work a great deal better. We would actually be able to do things that support learning, rather than that support credit scores; to give feedback that leads to improvement, rather than as a form of punishment or reward; to allow students to expand and explore pathways that diverge rather than converge; to get away from our needs and to concentrate on those of our students; to support people’s growth rather than to stunt it by setting false goals; to valorize creativity and ingenuity; to allow people to gain the skills they actually need rather than those we choose to teach; to empower them, rather than to become petty despots ourselves. And, in an entirely separate process of assessment that teachers may have little or anything to do with at all, we could enable multiple ways to demonstrate learning that are entirely dissociated from the process. Students might use evidence from learning activities we help them with as something to prove their competence, but our teaching would not be focused on that proof. It’s a crucial distinction that makes all the difference in the world.  This is not a revolutionary idea about credentialling – it’s exactly what many of the more successful and enlightened companies already do when hiring or promoting people: they look at the whole picture presented, take evidence from multiple sources, look at the things that matter in the context of application, and treat each individual as a human being with unique strengths, skills and weaknesses, given the evidence available. Credentials from institutions may be part of that right now, but there is no reason for that idea to persist and plenty of alternative ways of showing skills and knowledge that are becoming increasingly popular and significant, from social network recommendations to open badges to portfolios. In fact, we even have pockets of such processes well entrenched within universities. Traditional British PhDs, for example, while they are examined through the thesis and an oral exam (a challenging but flexible process), are examined on evidence that is completely unique to the individual student. Students may target the final assessment a bit, but the teaching itself is not much focused on that. Instead, it is on helping them to do what they want to do. And, of course, there are no grades involved at all – only feedback.

Conclusion

It’s going to be a long slow struggle to change the whole of the educational system across most of the world, especially as there’s a good portion of the world that would be delighted to have these kinds of problems in the first place. We need education before we can have cheating. But we do need to change this, and exams are a good place to start. It changed once before, with far less research to support the change, and far weaker technologies and communication to enable it. And it changed recently. In the grand scheme of things, the first ever university exam of the kind we now recognize as almost universal was the blink of an eye ago. The first ever written exam of the kind we use now (not counting a separate branch for the Chinese Civil Service that began a millenium before) was at the end of the 18th Century (the Cambridge Tripos) and it was only near the end of the 19th Century that written exams began to gain a serious foothold. This was within the lifetime of my grandparents. This is not a tradition steeped in history – it’s an invention that appeared long after the steam engine and only became significant as the internal combustion engine was born.  I just hope institutions like ours are not heading back down the tunnel or standing still, because those heading into the light are going to succeed while those that stay in the shadows will at best become the laughing stock of the world. 

On the subject of which, do watch the video. It is kind-of funny in a way, but the humour is very dark and deeply tragic. The absurdity makes me want to laugh but the reality of how this crazy system is wrecking people’s lives makes me want to cry. On balance, I am much more saddened and angered by it than amused. These are not bad people: this is a bad system. 

Reference

Davis, S., Drinan, P., and Gallant, T. (2009). Cheating in School: What We Know and What We Can Do. West Sussex, UK: Wiley-Blackwell.

Footnote

I know some people will want to respond that the threat or reward of assessment is somehow motivating. If you are one of those, this postscript is for you. 

I understand what you are saying. That is what many of us were taught to believe and it is one way we justify persisting despite the evidence that it doesn’t work very well. I agree that it is motivating, after a fashion, very much like paying someone to do something you want them to do, or hitting them if they don’t. Very much indeed. You can create an association between a reward/punishment and some other activity that you want your subject to perform and, as long as that association persists, you might actually make them do it. Personally speaking, I find that quite offensive, not to mention only mildly effective at achieving its own limited ends, but each to their own. But notice how you have replaced the interest in the activity with an interest in the reward and/or the desire to avoid punishment. Countless research studies from several fields have pretty conclusively shown that both reward and punishment are strongly antagonistic to intrinsic motivation and, in many cases, actually destroy it altogether. So, you can make someone do something by destroying their love of doing it – good job. But that doesn’t make a lot of sense to me, especially as what they have learned is presumably meant to be of ongoing value and interest, to help them in their lives. It is my belief that, if you want to teach effectively, you should never make people learn anything – you should support them in doing so if that is what they want to do. It is good to encourage and enthuse them so that they want to do it and can see the value – that’s a useful teacher role – but it’s a whole different ballgame altogether to coerce them. Alas, it is very hard to avoid it altogether until we change education, and that’s one good reason (I hope you agree) we need to do that.

For further information, you could do worse that to read pretty much anything by Alfie Kohn. If you are seeking a broader range of in-depth academic work, try the Self Determination Theory site.

Interview with Terry Anderson in The Voice Magazine – a Publication of the Athabasca University Students Union

Posted , updated on

The second part of an interview with Terry Anderson (with a link to the first part) talking about his approach to teaching, a bit about his research, a bit about some of his many contributions to AU, and a fair bit about the Landing and its value. Great stuff!

Terry voices his impatience with the slow uptake of the Landing, which is understandable. It is a bit frustrating to know that it can (and, for some, already does) radically improve the learning and working experience for staff and students alike, providing social presence, control, and a massive range of learning opportunities and activities that go far beyond anything possible in closed course containers or generic tools like mailing lists and project sites, but that we only see a ‘mere’ 7000+ users, 400+ groups, dozens of courses, as well as scores of posts and thousands of external visitors every day!  I have always taken the view that critical passion trumps critical mass every time. That is what keeps the Landing going and that is what will sustain it for the long haul. It’s a slow but steady burner that keeps getting better and more useful with every single post and comment, and the word is spreading: deeper, more sophisticated use is notably rising, diversity is increasing, and people are visiting longer and more frequently. For a system developed on a shoe-string, with very little central funding or support, it has done pretty well for itself for well over 5 years and I hope and expect it will continue to do so for the foreseeable future. And it does continue to evolve – you can expect to see ongoing improvements, not just tweaks but major gains in functionality, over the coming months, and we are working on a major upgrade to the underlying software framework at the same time that should enable even more improvements. With any luck we might even eventually get a bit more support from central pockets that might really let it fly but, even without much of that, the site grows in importance and value all the time.

Address of the bookmark: http://www.voicemagazine.org/articles/featuredisplay.php?ART=10321

Understanding transactional distance in web-based learning environments: An empirical study – Huang – 2015 – British Journal of Educational Technology – Wiley Online Library

Posted , updated on

This is a very important paper examining and verifying Moore’s theory of transactional distance. Sadly it is not open but those with AU accounts will be able to access it via AU Library.

The paper mostly confirms an inverse relationship between dialogue and transactional distance (more dialogue reduces the distance), and a direct relationship between structure and transactional distance (more structure increases the distance), that autonomy tends to slightly reduce the distance, and that a lack of both structure and dialogue is very bad indeed. But it also provides an interestingly nuanced view that appears to show high levels of both structure and dialogue are better still, as Moore himself predicts. I’m a bit sad that they used one of my earlier papers rather than my later book, Control & Constraint in E-Learning, because the findings strongly confirm my thesis in that book that modern forms of social media make it possible to have both high structure and high dialogue and that, consequently, transactional distance is lowest of all in such circumstances.  This paper appears to show that best of all worlds appears to occur when students use blogs, wikis, Twitter, etc, with notably lower transactional distance reported when compared with email/discussion forums.

There are also some interesting correlations between age and perceived transactional distance that the authors (I think rightly) put down to greater autonomy – older students tend to perceive lower transactional distance and tend to be more autonomous. There are also unexplained correlations with ethnicity that I am as puzzled about as the authors – this is intriguing and demands further investigation. I suspect cultural factors may explain this rather than ethnicity per se.

Though the studies are well conducted, I have some concerns with the measures and definitions used: I am particularly bothered by the split of interactions between people, content and interfaces. Structure here is defined by higher levels of learner-content and learner-interface interactions. I have always found ‘interface’ to be a very puzzling distinction as it is the interface that mediates both dialogue and learner-content interactions, so it is neutral to structure or dialogue – it ain’t what it is, it is how it is done that matters. It is precisely that neutrality that makes the value of social media so high because, out of dialogue, structure in the interface can emerge and/or the structure of the interface can help to shape dialogue. It is also interesting that transactional distance is (quite rightly) not just seen as a distance between instructor and student but between students and students: this is correct because students are one another’s teachers too. Again, this helps to explain the increased value of modern social media in reducing transactional distance because dialogue becomes content in the process, in ways that it does not in discussion forums and email.

Address of the bookmark: http://onlinelibrary.wiley.com/doi/10.1111/bjet.12263/abstract?systemMessage=Wiley+Online+Library+will+be+disrupted+on+21st+March+from+10%3A30+GMT+%2806%3A30+EDT%29+for+up+to+six+hours+for+essential+maintenance.++Apologies+for+the+inconvenience.&userIsAuthenticated=false&deniedAccessCustomisedMessage=

Ivy League For Free: What One Man Learned By Crashing Elite Colleges For 4 Years

Posted , updated on

A thought-provoking article on a man who gate-crashed some top tier US university courses as well as here in Canada. This was both politically and personally motivated. The man not only attended lectures and seminars but also got access to networks, direct interaction with professors, and received many of the benefits of a traditional university education bar the accreditation and support.

Attendance at lectures is rarely monitored and it has long been touted as one of the best free entertainment options available, if you choose your lectures wisely. I think I first recall reading about it in Playpower, a book on alternative culture from the early 70s. As the article suggests, it is generally pretty well accepted in many institutions and, as long as it doesn’t harm paying students (in this case I’d guess it was actually beneficial), it is hard to see why anyone would object. It might be a little trickier to get away with attending small tutorials or workshops involving restricted resources (e.g needing logins to university computers), and it would not always be easy to get the accompanying documentation and schedules, but this seems eminently do-able for many courses. I don’t think it scales well – most universities would probably start to institute controls if a large number of non-paying students started to join in, especially if they used up rival resources like handouts or materials, or hog tutor time in ways that harmed others. In some subjects it would not work at all. But it is quite an interesting perspective on openness. Face-to-face can be open too – in fact, it might be the default.

Makes me wonder about closed online courses and open MOOCs. Are we really as open as we claim? I think not so much.

Address of the bookmark: http://www.fastcompany.com/3043053/my-creative-life/ivy-league-free-what-one-man-learned-by-crashing-elite-colleges-for-4-years

Four Reasons to Worry About "Personalized Learning" – Alfie Kohn

Posted , updated on

Alfie Kohn is one of my favourite writers and this is a good example of why. Kohn’s basic concern with personalized learning is that  “Each of us can do what he likes as long as he ends up fundamentally similar to everyone else” but he drills down a lot further. These are his fundamental worries with how personalized learning is being conceived and implemented:

1. The tasks have been personalized for kids, not created by them.

2. Education is about the transmission of bits of information, not the construction of meaning.

3. The main objective is just to raise test scores.

4. It’s all about the tech.

I share Kohn’s concerns – I am appalled by a lot of the adaptive hypermedia, learning analytics and intelligent tutoring systems that I have seen, dating back around three decades or more but apparently now reaching critical mass in the personalization movement. He is absolutely right to be afraid of how these things are being used and implemented not just for kids but for adults. The trend towards measurability, uniformity and control of students is a heinous crime against this and future generations, made much worse by the institutionalized power abuse that comes with the package.

On the other hand, I think that it doesn’t have to be that way. First of all, it is important to consider the granularity: adaptive systems can have a lot of value when they are available in small pieces that give learners the power to choose them as tools to support their personal learning (and, as Kohn points out, social learning). I am also not averse, on the whole, to tools that suggest and advise but that do not dictate. But mainly, like all things technological, from painting to bridge construction, it ain’t what you do, it’s the way that you do it that matters. It’s a problem when the process of learning is hardened into a toolset that is dictated by a small hierarchy and not socially negotiated. It’s the higher levels of control that are the most worrisome technologies in this assembly: the tools just make it easier to exert that control. We should fight against the standardized tests, the teaching for grades, the manipulation of learners for political ends. But, when that toolset simply adds to what we already have, it can make the whole system more creative, flexible and controllable by learners. It’s the difference between what Franklin calls ‘holistic’ and ‘prescriptive’ technologies.

Address of the bookmark: http://www.alfiekohn.org/blogs/four-reasons-worry-personalized-learning

Laws of the academic jungle

Posted , updated on

My former VC, Sir David Watson, who died yesterday after a short illness, was a gentle, wise and caring man from whom I learned much and who supported me, guided me and challenged me in myriad ways. He was a remarkable man: a great educator, a fine musician and a gifted leader. I believe that he knew not just the names of all his 2500+ staff, but also enough about each and every one of them to sustain a conversation about their interests, achievements and friends. He was modest enough to deny this but, in 15 years of working with him, I never saw him falter and nor did anyone I ever spoke with about it. It was a phenomenon.

I share his nine Laws of Academic Life (AKA Laws of the Academic Jungle) in fond remembrance…

  • Academics grow in confidence the farther away they are from their true fields of expertise (what you really know about is provisional and ambiguous, what other people do is clear-cut and usually wrong)
  • You should never go to a school or department for anything that is in its title (which university consults its architecture department on the estate, or – heaven forbid – its business school on the budget?)
  • The first thing a committee member says is the exact opposite of what she means (“I’d like to agree with everything the vice-chancellor has just said, but…”; or “with respect”…; or even “briefly”)
  • Courtesy is a one-way street (social-academic language is full of hyperbole, and one result is the confusion of rudeness – or even cruelty – with forthrightness; however, if a manager responds in kind, it’s a federal case)
  • On email, nobody ever has the last word
  • Somebody always does it better elsewhere (because they are better supported)
  • Feedback counts only if I agree with it
  • The temptation to say “I told you so” is irresistible
  • Finally, there is never enough money, but there used to be.
 
Sad to have lost one of the great educational leaders.

Address of the bookmark: http://www.timeshighereducation.co.uk/news/laws-of-the-academic-jungle/408835.article

Transactional Distance among Open University Students: How Does it Affect the Learning Process? : European Journal of Open, Distance and E-Learning

Posted , updated on

Interesting study looking into transactional distance between online learners at a Greek open university, with some great qualitative findings.

The findings are very revealing about the role and nature of dialogue in online learning at the authors’ university. As we noted in our book, Teaching Crowds, transactional distance becomes very complex once there are multiple ‘teachers’ (or teaching presences) involved, where peer and content interactions are multi-dimensional and so transactional distance shifts and varies all the time. The study reveals some quite nuanced and differentiated communication patterns that demonstrate this quite nicely.  A bit of fuzziness shows through, however, where what is reported is mainly levels of communication rather than perceived transactional distance. The two are very closely related, inasmuch as communication is a necessary but not sufficient condition for reducing transactional distance, but they are not the same thing. 

I find it hard to imagine, as suggested for future study in the conclusion, what ways one might measure transactional distance in learner-content or learner-interface interactions that would not make the distance extraordinarily high. This is almost true by definition, apart from in ‘creepy’ ways (e.g. if the learners felt psychological closeness and attachment with an AI) or, maybe stretching the definition a bit, through guided didactic conversation. I will be interested to see how the writers address this!

Address of the bookmark: http://www.degruyter.com/view/j/eurodl.2014.17.issue-1/eurodl-2014-0002/eurodl-2014-0002.xml

For Sale: “Your Name Here” in a Prestigious Science Journal

Posted , updated on

A Scientific American article on the prevalence of plagiarism and contract cheating in journal articles.  The tl;dr version lies near the end of the article:

“Now that a number of companies have figured out how to make money off of scientific misconduct, that presumption of honesty is in danger of becoming an anachronism. ‘The whole system of peer review works on the basis of trust,’ Pattinson says. ‘Once that is damaged, it is very difficult for the peer review system to deal with.'” 

Very sad. The only heartening thing about all this is that there are now thousands of scam journals (I think I now get at least half a dozen solicitations from these every day that I have learned to junk immediately) who would be more than willing to publish such articles. I rather like the idea that worse than useless fraudulent articles might get published in worse than useless scam journals. A nice little self-contained economy. Unfortunately, some of the cheats target real journals with real reputations and, worse, may be believed by genuine researchers who are taken in by the lies they purvey, endangering the whole academic research endeavour. Apparently the going price for that in China is around 93,000RMB, or $15,000.

This is very much like the issue we face in course assessment too. In some of my own courses I have designed what I reckon to be virtually foolproof methods of preventing most forms of cheating. They mostly work pretty well, but they don’t cope much better with contract cheating than more traditional assignment/exam based courses. My only partial solution to that problem is to try to price cheats out of the market: most of my courses have to be done from start to finish in order to pass, which is a lot more time consuming than writing a few boilerplate essays, exams or exercises. For assignments and exams on most courses you can get a passing grade for as little as $5, if you are willing to take the risk. The risk of discovery is very high because the essay mills tend to plagiarize or self-plagiarize (well, they are cheats – caveat emptor!) and, due to the semi-public nature of cheating sites, it is just as easy for us to discover students seeking ghost writers as it is for them to seek a ghost writer. In fact, when we find such sites, we tend to pass on our findings to colleagues in other institutions, a nice example of informal crowd-sourcing. However, I am absolutely sure some do get away with it, and it makes little or no difference whether teaching is online or face to face. There’s an example of contract cheating in exams in today’s news, but it is hardly newsworthy, apart from that it is endemic. Beyond contract cheating, I also know that some students have family members or friends who are motivated to ‘help’, sometimes quite considerably. There was a charmingly improbable example of a mother sitting her daughter’s exam a while back, for instance. 

I suspect that the ultimate solution to this in the case of courses is structural, not technological nor even directly pedagogical. We are in an un-winnable arms war in which everyone loses as long as the purpose of courses is seen to be to get accreditation, rather than to enable learning. As long as a grade sits enticingly at the end of it, that will inevitably cause some students to seek shortcuts to getting it. Cheats destroy the credibility not just of their own qualifications but those of every other student who has honestly run the course. If we got rid of grades altogether, cheating during the learning process would dry up to the merest trickle (though, bizarrely, might not go away altogether). Making accreditation a separate issue, completely disassociated from learning and teaching, would allow us to concentrate our firepower on preventing cheating at the point of accreditation rather than distracting us during a course, so we could make our courses far more engaging, enjoyable and useful: we could simply concentrate on pedagogy rather than trying to design cheating out of them. For the (entirely separate) accreditation, we could let rip with all the weaponry at our disposal, of course: biometrics, Faraday cages, style detectors, plagiarism detection tools and all the multifarious technologies and techniques we have developed to attempt to thwart cheats could be employed with relative ease by specialists trained to spot miscreants. Better still, we could use other means of proving authenticity such as social network analysis combined with public facing posts, or employer reports, or authentic portfolios created over long periods with multiple sources of authentication. This would also have the enormous benefit of largely solving what is perhaps the biggest challenge in all of education, that of motivation, getting rid of the extrinsic driver that eats at the soul of learning in our educational systems. It would also allow learners to control how, when, with whom and what they learn, rather than having to take a course that might bore them or confuse them. They could easily take a course elsewhere – even a MOOC – and prove their knowledge separately. It would make it easier for us to design courses that are apt for the learning need, rather than having to fit everything into one uniform size and shape. It would also overcome the insane contradiction of teachers telling students they have failed to learn when, quite clearly, it is the teachers that have failed to teach. Athabasca does, of course, have the mechanisms for this, in its PLAR and challenge processes. It could easily be done.

A similar solution might work, at least a little, for journal cheaters. There are different cultural norms around cheating in China, as I have observed previously, that perhaps play a role in the preponderance of Chinese culprits mentioned in the article, but a lot of the problem might be put down to the over-valuation of publication for career progression, prestige and reward in that country. If the rewards and reputation were less tightly bound to publication and more intrinsic to the process, we might see some improvement. This could be done in many ways: for instance, greater value could be given to internal dissemination of results,  open publication (inherently less liable to fraud thanks to many eyes), team work, blogging, supervisor reports, peer review (of people, not papers) and citations (though that is inevitably going to be the next easy target for fraud, if it is not already, so should not be treated too seriously). There are lots of ways to measure academic value apart from through numbers of publications, many of which relate to hard-to-spoof process rather than an easily forged product. The worrisome trend of journals charging authors for publication is an extremely bad idea that can only exacerbate the problem: publication becomes a commodity that is bought and sold, of value in and of itself (like grades) rather than as a medium to disseminate research.

These are sad times for academia, eaten from the inside and out, but they also present an opportunity for us to rethink the process. The standards and values that have evolved over many centuries and that once stood us in good stead when adult education was an elite affair just don’t apply any more. What our forebears sought in opening up academia was to expand the reach of education to all. Instead, we turned it into a system to deliver accreditation. That system is on a self-destruct course as long as we continue to act as though nothing has really changed. 

Address of the bookmark: http://www.scientificamerican.com/article/for-sale-your-name-here-in-a-prestigious-science-journal/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%253A+ScientificAmerican-News+%2528Content%253A+News%2529

Defaults matter

Posted , updated on

I have often written about the subtle and not-so-subtle constraints of learning management systems (LMSs) that channel teaching down a limited number of paths, and so impose implicit pedagogies on us that may be highly counter productive and dissuade us from teaching well – this paper is an early expression of my thoughts on the matter. I came across another example today.

When a teacher enters comments on assignments in Moodle (and in most LMSs), it is a one-time, one-way publication event. The student gets a notification and that’s it. While it is perfectly possible for a dialogue to continue via email or internal messaging, or to avoid having to use such a system altogether, or to overlay processes on top of it to soften the hard structure of the tool, the design of the software makes it quite clear this is not expected or normal. At best, it is treated as a separate process. The design of such an assignment submission system is entirely about delivering a final judgement. It is a tacit assertion of teacher power. The most we can do to subvert that in Moodle is to return an assignment for resubmission, but that carries its own meanings and, on resubmission, still returns us to the same single feedback box.

Defaults are very powerful things that profoundly shape how we behave (e.g. see here, here and here). Imagine how different the process would be if the comment box were, by default, part of a dialogue, inviting response from the student. Imagine how different it would be if the student could respond by submitting a new version (not replacing the old) or by posting amendments in a further submission, to keep going until it is just right, not as a process of replacement but of evolution and augmentation. You might think of this as being something like a journal submission system, where revisions are made in response to reviewers until the article is acceptable. But we could go further. What if it were treated as a debugging process, using approaches like those in Bugzilla or Github to track down issues and refine solutions until they were as good as they could be, incorporating feedback and help from students and others on or beyond the course? It seems to me that, if we are serious about assignments as a formative means of helping someone to learn (and we should be), that’s what we should be doing. There is really no excuse, ever, for a committed student to get less than 100% in the end. If students are committed and willing to persist until they have learned what they come here to learn, it is not ever the students’ failure when they achieve less than the best: it is the teachers’.

This is, of course, one of the motivations behind the Landing. In part we built this site to enable pedagogies like this that do not fit the moulds that LMSs ever-so-subtly press us into. The Landing has its own set of constraints and assumptions, but it is an alternative and complementary set, albeit one that is designed to be soft and malleable in many more ways than a standard LMS. The point, though, is not that any one system is better than any other but that all of them embed pedagogical and process assumptions, some of which are inherently incompatible.

The solution is, I think, not to build a one-size-fits-all system. Yes, we could easily enough modify Moodle to behave the way I suggest and in myriad other ways (e.g. I’d love to see dialogue available in every component, to allow student-controlled spaces wherever we need them, to allow students to add to their own courses, etc) but that doesn’t work either. The more we pack in, the softer the system becomes, and so the harder it is to operate it effectively. Greater flexibility always comes at a high price, in cognitive load, technical difficulty and combinatorial complexity. Moreover, the more we make it suit one group of people, the less well it suits others. This is the nature of monolithic systems.

There are a few existing ways to greatly reduce this problem, without massive reinvention and disruption. One is to disaggregate the pieces. We could build the LMS out of interoperable blocks so that we could, for instance, replace the standard submission system with a different one, without impacting other parts of the system. That was the goal of OKI and the now-defunct E-Framework although, in both cases, assembly was almost always a centralized IT management function and not available to those who most needed it – students and teachers. Neither have really made it to the mainstream. Sakai (an also-ran LMS that still persists) continues to use OKI technologies under the hood but the e-framework (a far better idea) seems dead in the water. These were both great ideas. There just wasn’t the will or the money, and competition from incumbents like Moodle and Blackboard was too strong. Other widget-based methods (e.g. using Wookie) offer more hope, because they do not demand significant retooling of existing systems, but they are currently far from on the ascendent and the promising EU TENCompetence project that was a leader behind this seems moribund, its site offline.

Another approach is to use modules/plugins/building blocks within an existing system. However, this can be difficult or impossible to manage in a manner that delivers control to the end user without at the same time making it difficult for those that do not want or need such control, because LMSs are monoliths that have to address the needs of many people. Not everyone needs a big toolkit and, for many, it would actively make things worse if they had one. Judicious use of templates can help with that, but the real problem is that one size does not fit all. Also, it locks you in to a particular platform, making evolution dependent on designers whose goals may not align with how you want to teach.

Bearing that in mind, another way to cope with the problem is to use multiple independent systems bound by interoperability standards – LTI, OpenBadges or TinCan, for example. With such standards, different learning platforms can become part of the same federated environment, sharing data, processing, learning paths and so on, allowing records to be kept centrally while enabling incompatible pedagogies to run independently within each system. That seems to me to be the most sensible option right now. It’s still more complex for all concerned than taking the easy path, and it increases management burden as well as replicating too much functionality for no particularly good reason. But sometimes the easy path is the wrong one, and diversity drives growth and improvement.