Does the Online Environment Promote Plagiarism?

Posted , updated on

Executive summary: no.

Thanks to Terry Anderson for alerting me to Ison’s interesting and informative paper, which suggests there is no significant difference  between levels of plagiarism in doctoral dissertations/theses whether students are online or not. There are slightly different distributions – notably, students at physical institutions appear to be somewhat more prone to severe cases of plagiarism. I’d hazard a guess that this small variation has more to do with the demographic differences between online and face-to-face doctoral students rather than anything directly to do with modality. Distance learners tend to be a little older and a little more intrinsically motivated, on average, than their physically collocated counterparts. 

While it is, on the face of it, disturbing that more than half of the examined dissertations at a doctoral level (where most studies have shown that by far the least amount of cheating is normally to be found) appeared to have some level of plagiarism, the results should be treated with a generous pinch of salt. A lot of this revolves around:

  1. definitions. The authors note that the greyer area of self-plagiarism affects these results. It is worth remembering that, in many countries, it is not just accepted but positively required that doctoral students use their published work as part of their theses. Indeed, in many countries, such publications often make up by far the majority of the thesis. Even where that is not the case, it is normal to include published papers in appendices and it would be extremely unusual for a student not to at least partially re-use their doctoral work as a basis for papers and vice versa. It is a widespread and accepted practice that I think should be encouraged, not damned. There is not much better proof of research competency than publication in peer-reviewed journals and, as that competence is what a doctorate is supposed to show, it is churlish to exclude such evidence. It is also worth remembering that there are very few fully online doctoral programs. Even at Athabasca, which is about as extreme as it gets, almost all doctoral students get to spend a little face-to-face time with one another and their supervisors. Equally, there are very few fully face-to-face programs. Way back in the 1990s much of my supervisors’ help came to me online, even though they were only a minute down the hall. It’s just a matter of degree and perception.
  2. the effectiveness of TurnItIn as a plagiarism detector. Having used TurnItIn over many years, I have always found it necessary to look really closely at the passages that it identifies and never to take its scoring at face value, especially for those passages in the ‘yellow’ zone. It often fails to notice that verbatim or paraphrased passages have been correctly cited (or at least an honest attempt has been made), for instance. It can provide a useful alert to help narrow down the papers to be concerned about, it is usually pretty reliable when a lot falls into the red zone, and it can make preparation of evidence in a plagiarism case a great deal easier, but it is very far from infallible, producing many false positives and missing some quite blatant examples that have been lightly obfuscated. 

Whether or not the results are reliable at an individual or overall scale, the relative proportions are what is interesting here. The fact that there is little difference between levels of plagiarism for online and face to face learners is both unsurprising and heartening. 

 

Address of the bookmark: http://jolt.merlot.org/vol10no2/ison_0614.pdf

The trouble with (most) courses

Posted , updated on

I recently did a session at the University of Brighton’s Learning and Teaching Conference on the trouble with modules – the name used for what are more commonly known as ‘courses’ in North America, ‘units’ in Australia and ‘papers’ in New Zealand. A couple of people who missed the session have asked for more details than what was shown in the slides that I posted from the session, so this post is a summary of some of the main points. It is mostly gleaned from my notes that accompanied the short presentation part, tidied up and slightly expanded on a bit for the blog.  I have not gone into much detail about what would happen if we did away with courses altogether, nor described the results of any of the reflective activities that were involved in the original session as I have no notes on those parts and not enough time to write them. It does contain a bunch of ideas and suggestions about how to overcome some of the innate weaknesses of courses though, that I hope will have some value to somebody. If anything is unclear or arguable, I’m very happy to follow up via the comments on this post!

Why (most) courses are a bad idea

The taught university course as we know it today started out as nothing more than the study of a (single) book, in schools in pre-university times and in the early days of universities, nearly a thousand years ago. The master or lecturer would read the book and, perhaps, comment on it and discuss it with students. This made a lot of sense. Books were very expensive and rare objects, and so were scholars. It was by far the most efficient way to make use of a rival good (the teacher and/or the book) to reach as many people as possible. Whether or not it was the best way to learn, without it there would be no learning about or from the book at all. These efficiencies remained significant for the next 900 years or so after universities were invented (first in Bologna and, later, Paris, Oxford and the slow-moving flood that followed over the next few centuries, right up to the recent trend in MOOCs). The course slowly evolved into more subject-specific areas that often drew from many books and, later, papers, and the printing press made books slightly less of a luxury, but the general principle, that knowledge was thinly disributed and the most efficient way to make it available was one-to-many transmission in a physical room, continued to make sense. As universities grew, it was equally sensible that processes and architectures were designed to make this still more efficient. Timetables were used to schedule these scarce resources, lecture theatres designed to reach as many ears and eyes as possible, desks invented to take notes, blackboards invented to provide a source for them, written exams invented to make assessments easier to mark (the first were in 1789) and libraries and classification systems invented to store and retrieve books and periodicals. And, of course, if students and teachers were not around, there was no point in scheduling classes, so courses naturally divided around the holidays of Christmas, Easter and during harvest time in the summer, when (perhaps – this is disputed) students were called back to work on farms. All of this made perfect sense and made the best use of limited means – perhaps the only means that could have worked at all. And this is what we have inherited, whether or not we observe Christian holidays, whether or not we have almost free access to a cornucopia of information on the web and mobile devices, whether or not we have sophisticated information systems that make scheduling and organization of resources more flexible, or tools to connect us with anyone, anywhere, any time around the world. Around it we have built innumerable structures – notions of course equivalence that are related to accreditation and assessment, replicability, resource allocations, pay structures, etc – that have become very deeply embedded, not just within universities but in society as a whole. Universities have become gatekeepers that filter students as they come in and warrant their competencies as they leave, not just to become academics but to work in many occupations. And the unit of measurement is based around the course. Courses are so deeply embedded that, when people attempt educational reform, they are seldom even noticed, let alone questioned. If people want to make things better in education, they normally explicitly mean ‘better courses’. Even open and distance universities like Athabasca, that dumped prerequisites, the schedule and traditional lecture/tutorial/seminar format, adhere to the broad pattern of course length (measured now in hours of study, like most of the rest of the world outside North America) fixed outcomes and assessments.  Likewise, companies unwisely create or purchase courses for their employees to go out and learn stuff, albeit usually with fewer institutional constraints on timing, accreditation and format.  But there is no pedagogical reason whatsover that it should be this way.

What this means

The trouble is that courses, at least as they have mostly evolved, are not pedagogically neutral technologies. This is pretty obvious to anyone who has ever created one. It is a completely insane idea that every subject can be taught in multiples of precisely the same period or requires the same amount of study as every other. Typically (varying from place to place but usually unvaryingly within a given institution) this means 10-15 weeks or some multiple of that, or 100-200 hours of student effort. Taught courses, as we know them in our institutions today, have objectives and/or outcomes, and assessments to match, which conspire to mean that the intent is that everyone learns exactly the same thing or skill, whether they already know it, don’t need to know it, or not. Courses therefore differentiate – you pass them or fail them. Maybe you pass or fail them well or badly. As an incidental peculiarity, the blame for failure to teach is transferred to the students – they fail, not their teachers. This has big implications for an individual’s sense of self worth and on their ability to seek employment, and it impacts society (and individuals who suffer this process) deeply. Another consequence of this is that, thanks to the need for economies of scale and/or fitting things into timeslots or with other courses that might be similar, typically everyone is taught the same way on a given course, and taught the same things, whether or not it suits their needs, prior knowledge, interests and aspirations. While the notion of teaching to learning styles is palpable nonsense, there is no doubt that people have very different needs and preferences from one another, so parts of every course will bore or confuse some of their students some or all of the time and nearly all will contain parts of little or no relevance to a learner’s needs. None of this makes any pedagogical sense whatsoever. Bloom’s two-sigma problem (based on the fact that there is roughly a two sigma difference between results for those taught in traditional classrooms and those taught one-to-one) is a difficult challenge to address because, quite apart from their innate peculiarities, these features of the typical pattern followed by courses lead to one extremely big and elephant-in-the-room: they are inherently demotivating. 

Courses and motivation

People love to and need to learn, constantly and voraciously. It’s in our nature. If someone wants and/or needs to learn something, you have to do something pretty substantial to prevent them from doing so. Enter the taught course.

The first way that courses stand in the way of learning is, at first glance, relatively innocuous. The fixed nature and form of the course combined with its length necessarily means that, for the vast majority of students, parts will be boring, parts will be irrelevant, and parts will be over-taxing. This means most students’ need for challenge at an attainable level will not be met, at least some of the time.  It means that course content, process, rules of conduct, expectations and methods are strongly determined by someone else, sapping control away. Self-determination theory, a powerful construct that has been validated countless times over several decades, makes it very clear that, unless people feel in control, are challenged with achievable goals and experience relatedness, they will not be intrinsically motivated, no matter what other factors motivate them. Though often supporting relatedness (connection to something or someone beyond yourself), taught courses are, by and large, structured to reduce two of those three vital factors. It is no surprise then that teachers have to find ways to get around the lack of motivation engendered by the course format. There are a few teachers, sadly, who positively relish the exercise of their power, who enjoy rewarding and punishing students, who like to apply rigid control over behaviour in the classroom, who take a kind of sick pleasure in watching students suffer, who make students do things ‘because it’s for their own good’. They need our pity and support, but should not be allowed to teach until they have overcome this sickness. Luckily, by far the majority of us do our best to inspire, to actively encourage students to reflect on and actively align their intrinsic hopes and desires with what we are teaching, to offer flexibility and control, to empower students, to nurture their creativity, and to give some attention to each student. That’s the pleasure most of us get from teaching. We certainly don’t all succeed all of the time, even the best fail pretty regularly, and we could all improve, but at least we try. However, it’s an uphill battle.

This leads to the second and far more harmful effect of taught courses on motivation.  Most of us who work in higher education are constrained by the nature of the course and its accreditation to apply extrinsic rewards and punishments in the form of grades, even though we know it is a truly terrible idea. The reasoning behind the use of grades as motivators is understandable. We can easily observe that extrinsic methods do actually, on the whole, to some extent work, in the short term. Depending on the context, the effect can last from minutes to months. Indeed, behaviourists (who only ever did short-term studies) based a whole psychological movement on this idea. What is less obvious, and the most crucial structural disaster in the way the vast majority of courses are designed, is that they invariably and totally predictably utterly destroy any intrinsic motivation that people may already have, often irreparably. A big part of the reason for this is that it creates a locus of causality for a task or behaviour that is perceived as being controlled by someone or something else, so it does again come back to an issue of control, but this time the effects are devastating, not just reducing motivation but actively militating against it. This crowding effect has been demonstrated over and over again in well-designed and hard-to-refute research studies for decades. In many cases, rewards and punishments don’t even achieve what they set out to do in the first place. For example, companies that offer performance-related bonuses typically get lower performance from their workers, and daycare that punish parents who are late picking up their children find that parents actually pick them up even later. Worse, once the damage is done, it is very hard if not (sometimes) impossible to entirely undo it. It’s like the motivation pathways have been permanently short-circuited. Worse still, how we are taught is often a major factor in determining how we learn, and we come to expect and (like addicts) even depend on extrinsic motivation to drive us. This is one of the reasons I sometimes describe my role as ‘un-teaching’ – there is often a lifetime of awful learning habits to undo before we can even start. 

If you are not convinced, do check out a few of the hundreds of papers at http://www.selfdeterminationtheory.org/publications/ or read pretty much anything by Alfie Kohn, or Edward Deci, or Richard Ryan. There are plenty of studies from the field of education that look at the effects of rewards and punishments and find them worse than wanting.

Breaking the cycle

There are alternatives to typical institutional taught courses, some of them very common, others less so. The University of Brighton has a great program, the MSc/MA by Learning Objectives, in which students work with supervisors to develop a set of outcomes, a means of assessment, and a work plan to reach their goals. While there are a few time and process constraints here and there for practical reasons, they are not too onerous. Students on this program tend to pass it, not because its standards are low, but because everything is aligned with what they want and need to do. A few programs at Athabasca University have similarly flexible courses that act as a kind of catch-all to enable people to do things that matter to them.  PhD programs, of the traditional variety used in the UK, have (or had – the course-based American model is sadly becoming more prevalent) no obligatory courses and are entirely customized to and often by the individual student, with nothing but a few processes to ensure students remain on track and supported. They can take from 2-10 years to complete. This length can be a problem as our motivation usually changes over such a long time and extrinsic factors are often introduced that can affect it badly, but the general principle is a good one. Athabasca University’s challenge process makes it possible to completely separate accreditation from learning, which (almost) avoids the whole course problem altogether, though it does unfortunately only work if you happen to have the precise set of competences provided by actual taught courses. Its self-paced undergraduate courses, though still markedly constrained by a notional equivalence to their paced brethren, free students from the tyrrany of schedules, even if they do have other features that are overly limiting. PLAR/APEL processes that are common in institutions across the world separate learning from accreditation almost entirely. And that’s not to mention a huge host of teach-yourself methods and resources from Google Search to Wikipedia to the Khan Academy to Stack Exchange and hundreds of other fine online systems that most of us use when we actually want and need to learn something. And, of course, there are books, which have the great benefit of allowing us to skip things, re-read things, look up references and so on, so our paths through them are seldom linear and always under out control – unless we are forced to read them because of a course. 

But what about the run-of-the-mill?

Though there is much to be learned from existing methods that entirely or partially by-pass the harmful effects of taught courses, teachers in higher education operate under a set of ugly constraints that make it very difficult and often impossible for us to completely avoid their ill effects, especially when student numbers are large and things like professional standards bodies come into the picture. Until we achieve massive educational reform, which might allow us to provide multiple paths to achieving competence, that might separate learning from accreditation, that might be chunked in ways that suit the needs of learner and subject, we are mostly stuck with the offspring of a mediaeval system that has evolved to defend itself against change. Most of us have to grade things, we have  to make use of learning objectives/outcomes, and we don’t have much control over course length. Often, especially in lower-level courses and/or where standards bodies are involved, we have little control over the competences that need to be attained, whether or not we are competent to teach them. Moreover, many of the most effective existing methods of teaching without courses are very resource-hungry. It would be great to apply the (UK-style) PhD process to all of our teaching but it is economically infeasible. PhDs are expensive for a very good reason – many of the economic and physical constraints that drove the development of courses in the first place have not gone away, even though some have been notably diminished. Given these issues, I will finish this post with a few general ideas, suggestions and patterns to help reduce the ill effects of courses without destroying the system of which they are a part. 

Give control

Traditional teaching seems determined to take control away from learners, but we can do much to give it back. Amongst other things:

  • allow students to choose what they do and how they do it. For instance, I have a web development course that centres around a site that students build throughout the course, that is about something they choose and they care about, and a course process that encourages them to choose between (or discover for themselves or their peers) multiple resources and methods to learn the requisite skills along the way. It makes extensive use of peer support and encourages sharing of problems and solutions, so that students teach one another as a natural fall-out of the process. It uses reflection to support the process, and an assessment based on evidence (that the students select for themselves) of meeting specified learning outcomes. It’s far from perfect, and it does often cause problems (especially at first) for those who have learned dependence via our broken educational system, but it shows one way that learners can take the reins.
  • allow students to choose the learning outcomes. This is trickier to enact because of the rigid requirements we usually have to develop curricula and match them with those delivered elsewhere. However, if the outcomes we specify are not too specific, relating to broad competences, it is still possible to allow some flexibility to students to identify finer-grained outcomes that suit their needs and that are exemplars of the general overarching outcomes. I’ve found this approach easier to follow in graduate level courses in ill-defined subject areas – I don’t really have a way of doing this well for those that are constrained by disciplinary standards.
  • allow students to design their own assessments. This one is easier. Learning contracts are one way to do this, supported with scaffolding that allows students to develop their own plans for assessment. Similarly, we can ask for them to provide evidence in a form that suits them (one of the best computing assignments I have ever seen was mostly done as poetry, and I once had a great explanation of the ISO model of network management explained using Santa Claus’s elves). At the very least, we can offer alternative pre-written forms of assessment that students can choose between according to their preferences.
  • allow students to pick their own content. This is a trick I have used for several courses. I offer a menu of options that address the intended (broad) outcomes and negotiate which parts we/they will cover during the course. It takes a little more effort to prepare, but the payoff is large. For graduate level courses I sometimes encourage students to develop their own content that we all then use.
  • allow students to choose their own tools, media, platforms, etc. Where possible, students should not be limited in their choice of technologies needed to complete the course. This can be tricky where we are constrained by things like institutional platforms, but there are often ways to allow at least some flexibility (e.g. mobile-friendly versions, PDF and e-book formats, standard formats that allow the use of any editor or development tool, etc)
  • allow students to pick the time and place. This is the default at Athabasca University for most courses, but can be trickier when there are timetables and constraints of working with others according a schedule. Classroom flipping can help a bit, limiting what is done in the class to things that actually benefit from being somewhere with other people (feedback, dialogue, collaboration, problem-solving, etc), and leaving a lot to self-paced study. This is true online as well as in face-to-face teaching. Indeed, counter-intuitively, it is even one of the odd potential benefits of traditional lectures, inasmuch as they typically only take an hour of a student’s time once a week, between which students are free to learn as they please (not a completely serious point, but worth pointing out because of the important and universally applicable lesson it reminds us of, that teaching behaviours only have a tangential relationship with learning behaviours).
  • allow students to control social interaction. I am a huge fan of learning with other people but we all have different needs for engagement with others in our learning, and it doesn’t suit everyone equally all the time. Where possible, I try to build processes that let those that benefit from social interaction to work with others, but that let those that prefer a different approach to work alone, using evidence-based assessments rather than process-based ones. For instance, evidence can include help given to others or conversations with others, but can as easily come from individual work (unless social competences are on the menu for learning). I find it useful to build simple sharing (as opposed to dialogue) into the process so that even the least sociable of students share things and therefore support the learning of others.

Use better forms of extrinsic motivation

Extrinsic motivation is not all equally awful and some is barely distinguishable or even a part of intrinsic motivation. Extrinsic motivators lie on a spectrum from bad (externally imposed reward and punishment) to much better and more internally regulated varieties, such as:

  • doing things out of a sense of duty, guilt or obligation (introjected regulation) or, better,
  • doing things because they are perceived as worthwhile in themselves (identified regulation, e.g. losing weight) or, better still,
  • doing things because they are necessary steps to achieve something else we are really motivated to achieve (integrated regulation).

See http://www.selfdeterminationtheory.org/theory/ for more about these differentiations. There are plenty of ways to use this to our advantage. It can often, for instance, be useful to encourage reflection on a learning activity. This can be used to think about why we are doing something, how it relates to our needs and goals, and what it means to us. Reflection can kindle more effective forms of extrinsic motivation that are far less harmful than externally imposed rewards and punishments. It is also valuable to nurture community, so that students feel obligations to the team or to one another, and support one another when the going gets rougher. Also, seeing how others are motivated can inspire us to recognize similar motivations in ourselves. Shared reflections (e.g. via blogs) can be particularly valuable.

Grades are not always necessary. While getting rid of the need to summatively assess is seldom possible, we can often avoid the use of grades (pass/fail is a little better than a mark), and we can make it possible for students to keep at it without grading until it is right, thus reducing the chance of failure. My courses tend to have feedback opportunities scattered throughout but I explictly avoid giving any grades until the last possible moment. It can upset some students who have learned grade-dependence, so it is important that they are fully aware of the reasoning and intent, and that the feedback is good enough that they can judge for themselves how well they are doing (I don’t always get that bit right!). Of course, I am only suggesting that we lose the grades, not the useful feedback. Feedback is crucial to allowing students to feel in control – they need to know what they are doing well and what could be improved, and plentiful feedback can be hugely motivating, showing that other people care, contributing to a sense of achievement, and more. Good, descriptive feedback that focuses on the work (never the student) is a cornerstone of effective educational practice. Grades tell us little or nothing, while encouraging an extrinsic focus that is harmful to motivation.

Step outside the course

Making links beyond a single course can be very beneficial to motivation. I attended an interesting presentation (at the same conference this originated in) the other day by Norman Jackson who talks about lifewide as opposed to lifelong learning, an idea that captures this principle well. Creating opportunities for students to engage in external activities like (for example) clubs, societies, geological digs, competitions, community work, conferences, charitable work, kickstarters, Wikipedia articles, coding camps and so on can fill in a lot of motivational gaps, making it easier to see the relevance of a course, to feed new ideas into a course, to gain a greater sense of personal relevance and responsibility for one’s own learning, to expand on work done in a course in greater detail wihout the imposition of extrinsic motivation. Of course, students should be free to choose which of these they engage with and, better still, should find them for themselves. However, there is no harm in advertising such things, nor in designing courses that allow students to capitalize on learning from other activities within the course itself such as projects, show-and-tell sessions, flexible discussions and so on. There are also often opportunities for doing things across multiple courses, using outputs of one to feed another, or bringing together different skillsets for joint projects. Another way to reduce the harm slightly is to build multiple courses into a single overarching one, of lengths appropriate to the needs of the students and subject. 

Build learning communities and spaces rather than courses

Given the wealth of potential resources and people’s time that are available for free on the Web (not to mention in libraries) there is often no need to provide much, if any content (in the sense of stuff presenting subject matter). A couple of the most successful courses I have ever run have had no curriculum or content to speak of, just a set of broad outcomes, a very flexible and student-designed assessment, an approach to making use of the learning community and a responsive process to make it all happen. The process can take a surprising amount of time to develop, as it is important that it is both understood well by the students (including how it is assessed, expectations, norms, etc) and that it can be guaranteed to result in the intended outcomes (assuming these are not negotiated too). Getting that process and community right can be hard work both in the design phase and (especially) during the course but, when it does go right, it is very rewarding. I have often learned as much if not more than my students on those courses, and they are the only courses I have ever run with more than a couple of students where I have had nothing but grade A students (moderated by external examiners as well as by peers). The massive enthusiasm and passion that results from a rich learning community of learners who are in control of their own learning has to be seen to be believed. The essence of the method is to let go just enough but no more: a teacher’s role is to provide plentiful prodding, ideas, critical feedback and, above all, scaffolding so that students feel confident that they are making progress in useful directions (and get help when they are not). It is also a bit of a juggling act to make sure that even loose outcomes are met, especially as students tend to diverge in all sorts of different directions, some of which are brilliant and worth pursuing – getting those outcomes loose enough in the first place but sufficiently recognizable and relevant to academic careers is a bit of an art that I am still learning. It also takes a lot of energy and dedication to make it work so, if you are having a bad week or two, things can go topsy turvy pretty fast.  It is worth putting a huge amount of effort into the first few weeks, responding enthusiastically and personally at any time of day or night that you can afford in order to set the tone, show that you care, explain your approach and soothe any fears. Once you have established trust that you care, and have nurtured a strong learning community, students tend to help one another a lot and forgive you when you are less attentive later on. I try to design the process so that I can intentionally let go in later weeks too.

In conclusion

As an intrinsic design feature, traditional university taught courses and their attendant processes and regulations impose unnatural restrictions on both teachers and students, reducing control and stunting motivation. It would be great to throw off these restrictions altogether. We could make enormous gains simply through separating teaching from accreditation (at least, wherever possible – in extremely rare cases it really is true that there is only one person who can reliably judge competence and that person is the teacher). This may soon become a necessity rather than a virtue if MOOCs continue to evolve faster than the means to reliably accredit the results. Athabasca University already has the challenge process to cope with that, though is significantly fettered by the need to match competences achieved with those that apply to existing courses – our challenge process is insufficiently fine-grained to allow real flexibility. There would be equally great gains if we made courses the right size (typically though not necessarily small) to fit the needs of different students rather than shoehorning them to fit the needs of institutions. We have technologies than can take the hard work out of managing the ensuing complexity so traditional timetabling woes need not impede us, and it would make it much easier to mix and match, including to accredit learning done in different ways. However, there is plenty that can be done even within the constraints of a typical university course, as long as we are aware of the dangers and take steps to reduce the harm. I hope that this little piece and this smattering of suggestions has sparked an idea or two about how we might go about doing that. Perhaps, if more of us start to question the system and apply such ideas, it might help to make a climate where bigger change is possible. If you’re interested in finding out more, I have written about this kind of thing once or twice before, with slightly different emphases, such as at https://landing.athabascau.ca/blog/view/177831/the-monkeys-paw-effect-in-higher-education and at https://landing.athabascau.ca/blog/view/496760/cargo-cult-courses 

 

Two conferences in two days

Posted , updated on

I’ve just got back from a flying visit to the UK. The first thing I saw on arriving at the new and not at all unpleasant Heathrow Terminal 2 was Stephen Downes. Small world. We were getting luggage from different areas and lost each other in the rush to get to different places, but it was nice to see him, however briefly.

The main reasons I was in the UK were two conferences, The First European Conference on Social Media and the umpteenth Learning & Teaching Conference at the University of Brighton.  Sadly, they overlapped, which meant I only got to attend a day of each, but I managed to give two quite different sessions at both conferences. The first, at ECSM, was a traditional slide-based presentation about the Landing, why and how we built it, and what we might do differently if we started again. As an experiment, rather than my usual handful of images that sit behind most of my presentations, I threw nearly 50 slides (some with multiple build stages) at the stunned audience in 20 minutes. Quite fun. The second, at the L&T conference, was a much more discursive hour-long session  that questioned the fundamental notion of courses, which involved a few thought experiments and a lot of conversation among a very engaged crowd. 

ECSM was a very well-organized affair (disclaimer – the chairs were my friends Sue Greener and Asher Rospigliosi) which provided what I have hoped to see in a social media for some years but have previously been disappointed: diversity. When I put together my first social computing course a few years ago I tried to offer much the same kind of range as this conference provided, but have since been a bit worried that I was defining a discipline too early in its lifecycle. This is because most social media/social computing conferences I have been involved with over the past few years have fallen heavily into computer algorithm territory, which my course touches on but doesn’t make a central focus. I have sometimes thought that they would be better named as social network analysis conferences, as variations on that theme have totally dominated the proceedings. I have come across some social media conferences that drift entirely the other way, looking at social and sociological consequences, and a few that focus on a single subject area or context (education and/or learning being the ones that usually interest me most). In contrast, ECSM was delightfully broad, with offerings across the spectrum, with coverage that I feel vindicates my choice of subject matter and approach for a social computing course. It included a lot of papers related to business, politics, media, education and other general areas, and a wide range of research attitudes and methods from the highly algorithmic to the softest and fuzziest of media analyses and critical inquiries. There were plenty of case studies from lots of contexts and demonstrations or reports on plentiful interesting systems. I think this is a sign of a maturing area of study. Though they were not keynoting, I was impressed that the conference attracted the marvellous guru couple of Jenny Preece and Ben Schneiderman. My favourite discovery of the day was that Dutch police have a room in Habbohotel. At the conference dinner I sat next to John Traxler, who was doing the next day’s keynote (that I would miss). He continues to impress me as a creative and incisive thinker. We spoke more about beer, Brighton and music than mobile and social media, but it was fun.

I was not expecting as much out of the parochial Learning & Teaching conference the next day, but I was wrong. The first keynote by Sue Clegg on the arguable failure of widening participation was thought provoking and went down well. Though provocative, it was a bit dry for my taste – I’m not a fan of presentations read from sheets of notes. I’d rather read the notes and have a conversation. Its focus was also very UK-centric, which should have been interesting but I did not have sufficient background knowledge of the events and acronyms to which she referred. She also seemed unusually approving of higher education access rates in the US, ranking it highest in the world, which was more than a bit of a surprise to me: I guess it depends how you measure such things, but the OECD ranks the US well below Korea, Japan, Canada (we’re third!) and several European countries, including the UK, when it comes to higher education participation. None-the-less, her talk was mostly tightly argued and backed up by plentiful research. I had planned to leave and return to ECSM after my session, which followed Sue Clegg’s talk, but I was enjoying meeting old friends and sufficiently intrigued by later sessions to stay on. I am glad that I did, not just because it gave me a chance to catch up with old friends and colleagues.

The first presentation I saw was about use of the e-portfolio system Mahara for professional and personal development. The University of Brighton has a mature and well-implemented Mahara instance that is used for a great many things, from personal publication to coursework to CV writing. I was a bit sad to see that, in combination with a WordPress instance and a SharePoint system used by staff, it had pretty much replaced the innovative Elgg system, community@brighton, that was part of the inspiration for the Landing and that largely surpassed all three put together in functionality. After 8 or 9 years, the last few of those in a state of slow and painful decline, community@brighton is about to be decommissioned. Community@brighton was a little ahead of its time; it suffered greatly in an upgrade process after its first successful couple of years that resulted in the loss of a great deal of the network and communities that had thrived beforehand, and it never fully recovered the trust of its users; it was insufficiently diverse in its primary uses, being quite focused on teaching and, in its latter years, finding shared local accommodation; and it was not helped that its introduction coincided with the massive rise of Facebook (before most people realised how evil that site was). But it was a great system that was (and even as it nears exinction, possibly still is) the world’s largest social media site in an HE institution and a lot of innovative work was done on and through it.

I was interested to learn that the University of Brighton has outsourced its Mahara, Blackboard and some other systems to the cloud. Mahara runs on Amazon’s Cloud service and is managed by Catalyst IT, ( www.catalyst-eu.net) the company behind Mahara, all for around £12,000 (roughly $CAD20,000) per year, plus fairly minimal cloud charges. This seems pretty good value to me – very hard for an internal IS team to compete with that. Similarly, though Blackboard is the work of the devil and the costs are astronomical, moving away from Blackboard would be very difficult for the University of Brighton. This is thanks to the massive investment in materials and training already sunk into it, combined with Blackboard’s strenuous efforts to encourage that dependency and notoriously bad tools for getting data out. Bearing that in mind, it makes sense for the University to move to a hosted solution, especially given the terrible performance, countless bugs, regular and irregular downtime, and the large amount of effort needed to keep it running and to answer technical problems. At least it should now perform reasonably, get timely updates, rarely go down and just work, most of the time. On a cautionary note I was, however, intrigued to learn that the university’s outsourcing of student email (to Microsoft’s Irish branch – Google was rejected due to lack of adherence to European data protection laws) had met with an unfortunate disaster, inasmuch as Microsoft changed the terms and conditions that had formerly meant students would have an email address for life, to a much more limited term. Outsourcing is fine when it works, but it always depends on another company with very different goals than one’s own. I normally prefer to keep things in-house, despite the cost. It means that you retain control of the data no matter what and, just as importantly, the knowledge to use it.

After a very fine lunch, I attended a double-length session reporting on the University of Brighton’s findings and work resulting from the very large Higher Education Academy ‘What Works’ research initiative. ‘What Works’ was focused on improving retention rates, seeking reasons for students giving up on courses and programs, and seeking ways to help them succeed. Brighton was one of the 22 institutions involved in the £1M study. A large team from Brighton gave a very lively and highly informative sequence of presentations on the background, the research and the various interventions that had been attempted following the study, not all with equal success, but all of them interesting. The huge take-home for me was the crucial importance of a culture of belonging. This was singled out in the HEA research that fed into this as the most significant factor in determining whether or not a student continues. Other factors are closely related to this – supportive peers, meaningful interactions, developing knowledge and confidence, and relevance to future goals, and all contribute to belongingness. There are also other factors like perseverence, engagement and internalization that play a role. It is intriguing to me that the research into this started with something of a blank slate, and did not draw significantly on the extensive literature on motivation outside of an educational setting. If it had done, they would probably have identified control as a major factor too although, given the context (traditional educational systems are not great for giving students control, especially to those in their early months of study), it is not surprising that it was missed. In recent years I have typically followed self-determination theory‘s vocabulary of ‘relatedness’ for this aspect of motivation, but ‘belonging’ is a far better word that captures a lot of what is distinctive about the nature and value of traditional academic communities and practices. Significantly for me, that is something which we at Athabasca University tend not to do so well. With self-paced courses, a large number of visiting students and relatively limited communication tools (apart from the Landing, of course!) it is very hard for us to build that sense of belonging. When tutoring works well, it goes quite a long way to achieving it and occasionally a bit of community develops via Moodle discussions but, apart from the Landing, we do nothing much to support a wider sense of belonging. At least, not in undergraduate programs. I think we tend to do it fairly well in graduate programs, where it is easier to build more personal relationships, peer support and cohorts into the system. I intend to follow this up and explore more of the background research that led to the HEA team’s conclusions. 

The afternoon ended with Pimms, but not before a closing keynote by Norman Jackson on life-wide (as distinct from life-long) learning. I found the notion of lifewide learning pleasing, concentrating on a person’s whole learning life, of which intentional academic behaviour is just a small part. The idea is related to the notion of learning trajectories as posited by Michael Eraut, with whom Jackson has worked. There was lots to like in his talk, and it drew attention away from the very course-centric view that underpins much university thinking, and that I had criticized in my own session. He had lots of nice examples based on studies and interviews with students, none of whom simply followed a ‘course’, though perhaps the examples were a little too glibly chosen – this was appreciative enquiry. He also placed a great onus on his version of ‘learning ecologies’ to describe the lifewide process. His definition of a learning ecology differs considerably from mine, and others who have used the term. As far as I could tell, the focus was very much on an individual, and his definition of a ‘learning ecology’ related to the various things that individuals do to support their learning. This is not a very rich ecology! I think that simply means that we tend to do a lot of things when learning that affect our learning in other things, all in a richly connected self-nourishing fashion. While he did, when questioned, agree that there was much richness to be gained from ‘overlapping’ ecologies and learning with and from others, I don’t think he sees the overlap as anything more than that. For me and, I think, most others who have used the term, a learning ecology has emergent patterns and behaviours that are quite different from its parts, full of rich self-organization, and it is crucial to negotiating meaning and creating knowledge in a social context. In a learning ecology, everyone’s learning affects everyone else’s, with positive and negative feedback loops creating knowledge that goes far beyond what any individual could develop alone. 

I am back in Canada now and trying to catch up with the load of things that two conferences inevitably delayed. I usually reckon that a conference takes up at least three times the time taken by the conference travel itself – preparation and recovery time are always a significant factor. In fact, it should take longer to recover because it would be great to reflect further to help consolidate and connect the learning that inevitably happens during the intensive sessions and conversations that characterize conferences: too many learning opportunities are lost when we rush back into a pile of over-delayed work after such things. At the very least, posts like this are a necessity to help make sense of it all, not an optional extra, but there is a lot more that I would like to follow up on if I had the time. It is also a pity because the weather in Vancouver is stunning (maybe too hot and dry) and I have a newly purchased but very old boat floating outside that keeps calling me. 

 

Widening access, student retention and success national programmes

Posted , updated on

Another recommendation from a recent conference presentation that I attended in the UK. The Higher Education Academy of the UK commissioned this large-scale study and intervention to explore factors affecting retention and engagement in UK universities. The full report is at Building student engagement and belonging in higher Education at a time of change: Final report from the What Works? Student Retention & Success programme but this page leads to a useful set of summaries and recommendations that are a bit more easily digested. The summary report is great.

Amongst the key issues that impressed me are (original emphases):

“At the heart of successful retention and success is a strong sense of belonging in HE for all students. This is most effectively nurtured through mainstream activities that all students participate in. “

“Specific interventions cannot be recommended over and above each other. Rather the institution, department, programme and module should all nurture a culture of belonging through the way they function and relate to people. “

“Student belonging is achieved through:

  • Supportive peer relations

  • Meaningful interaction between staff and students

  • Developing knowledge, confidence and identity as successful HE learners

  • An HE experience relevant to students’ interests and future goals “

     

I really like the notion of a ‘culture of belonging’ and the holistic approach recommended in this report. At Athabasca University we do well in some of these areas but less well in others. I think we are often over-focused on subjects and specific competences, especially in undergraduate programs, to the exclusion of other vital pieces of the educational machine, which greatly inhibits the sense of belonging that the HEA project identifies as so central. We lose too many students before they even start. Though we tend to be at least on par with other institutions for keeping them on specific courses once they have submitted their first assignments, we don’t have as many moving on through programs as we might. But this is just symptomatic of a broader malaise, that it is very hard to feel a part of a learning community in our isolated online spaces. Interactions tend to be limited to tutor-student communication much of the time, and the various tools (notably Moodle) that we use for teaching are intentionally isolated from one another. There’s little cohesion or sense of the broader community, and not much that is obvious that we can feel we can belong to.  

All of this helps to explain some of the key motivations behind why we created the Landing. It is meant as a space where academic identities can be explored, reflected upon and discovered, where we can feel that we belong to a real and vibrant community, where we can meet peers, see how others think and learn, and engage in meaningful interactions with them. It’s kind of like a virtual campus or learning commons, a space where many things happen, people meet, post information, engage in dialogue.

The Landing has been a success in very many ways and has helped many (including me) to achieve a greater sense of belonging. In retrospect, though, it would have been better to have seamlessly built its social richness and conrollable engagement into all of our other systems rather than as yet another loosely linked monolith. This is not impossible to retrofit. Since earlier this year it has been possible to integrate the Landing fairly well with other pages, such as those provided via Moodle, through its embedding functionality (simply make an iframe and add ‘?view=embed’ to most Landing URLs to separate a post from the surrounding site). This is, however, still not as seamless as it should be and requires some skill and deliberate intent on the part of people embedding it. We’ve not seen much uptake yet, though we have not promoted it at all actively, and we probably should. The Elgg technology behind the Landing does, however, have the capability of being embedded far more deeply as a web service to other applications, so that it can appear to be part of a quite different site (Moodle, say, or the main website, or MyAU, or pretty much anything). To make this happen needs a lot of carefully coordinated effort and clear communication between developers and managers of disparate systems that might use such a service, and plenty of planning, so it is not trivial to do. I’d be interested in doing such a project that either built on the Landing (makes sense, especially as it has other roles too) or started afresh to embed cross-cutting social engagement, sharing, connection and communication into all our student-facing sites, with all the same features and strengths (like discretionary access control, persistence, ownership, etc) of the Landing but without the need to go out of your way to visit it. I think I feel a research proposal coming on.

 

Address of the bookmark: http://www.heacademy.ac.uk/what-works-retention

"Future time orientation predicts academic engagement among first-year " by Louise Horstmanshof and Craig Zimitat

Posted , updated on

A study recommended during a recent conference that I attended that looks into student retention and engagement as a function of temporal orientation. In brief, Time Perspective (TP) theory predicts that

“students who are confident of their abilities (Past Positive) and who believe that their efforts produce results (low Present Hedonistic) are more likely than those who do not, to work towards a future goal (Future) to which they are committed and with which they can identify. Thus, by harnessing their time perspectives, they are able to regulate their behaviour to persist with their studies to achieve their educational goals”

The well-conducted study confirms the hypothesis but goes into a lot more detail, differentiates issues much further, and comes to some sophisticated conclusions that show that it is interestingly complicated. There is no silver bullet, behaviours are hugely interdependent and contextually situated, and multiple and diverse intervention strategies are needed to support students on their learning journeys through a university. It also provides some useful hints about how to help students improve their chances. It’s worth reading if you have any involvement with education, whether as a teacher or a learner. If you get overwhelmed by the tabular representations of the results of the study (thorough but turgid), after reading the theoretical background, skip to the discussion and implications sections.

Address of the bookmark: http://epubs.scu.edu.au/tlc_pubs/194/

There's a fitness tracker for your vagina. Quantifying your life has gone too far

Posted , updated on

A Guardian article from  Jess Zimmerman. The arguments are inelligent and Zimmerman recognizes the value as well as the dangers of socially-enabled biofeedback devices. The kGoal (tagline ‘Fitbit for your vagina’) actually sounds like rather a good idea, but the cons are significant. I particularly like “But the pitfall of data devices … is that they hijack your reward pathways” and “The quantified self … takes theaggregate self out of the equation”. Good food for thought, and some important lessons for those seeking to gamify many things, including learning and teaching.

Address of the bookmark: http://www.theguardian.com/commentisfree/2014/jul/14/fitness-tracker-vagina-quantified-life

The Distant Crowd: Transactional Distance and New Social Media Literacies

Posted , updated on

This is a paper for IJLM that I wrote with Terry Anderson, exploring how the distributed nature of teaching and learning in social technologies significantly messes with Michael Moore’s theory of transactional distance in all sorts of interesting ways. In it, we ponder on the literacies learners need in order to take best advantage of social media; we describe the different social forms of groups, nets and sets, and the emergent collectives that develop around them, that together form the backbone of our forthcoming book;  and we discuss different kinds of teaching presence that emerge in each form, suggesting ways of addressing the potential lack of reliability and credibility when the teacher (and thus transactional distance) is distributed (in a net), anonymous (in a set), or emergent (in a collective).

According to the date of the special issue of which it is a part, MIT Press impressively managed to publish this paper nearly two years before we finished writing it. It is true that we sent them the first draft in 2012 but it was not actually published till this month. Hard to know how to cite this.

One of the things that the paper mentions is that learning through networks can, under the right conditions, be more effective, timely and relevant than traditional group-oriented methods. This was rather delightfully and self-referentially brought home to me by the fact that I learned of the paper’s actual publication (as opposed to predicted date) via Twitter. 

Address of the bookmark: http://www.mitpressjournals.org/doi/abs/10.1162/IJLM_a_00104#.U7Hmexb07EJ

LearningLocker LRS – version 1.0 now available

Posted , updated on

Learning Locker has left beta and is now at version 1.0. This may be a significant milestone in a series of developments that could profoundly affect the future of online learning and perhaps the whole educational system.  

Learning Locker is an open source implementation of a TinCan (xAPI) learning record store (LRS). It provides a repository to record information about learning activities and outcomes, using open standards for import and export. That’s about it: some sorting and search tools, some export facilities, and the means to store information about learning from other applications. While other xAPI LRSs already exist, this open source implementation seems the most promising so far, the most feature-complete, and the most likely to see widespread adoption. OK, unless you are a learning technology geek I realise that this might sound rather dull and arcane, but the potential for disruption, especially given widespread support for the experience API (xAPI) standard in a wide range of applications, is quite high. Amongst other things:

  • it is a critical part of the infrastructure to free us from the mediocre monolithic silos of learning management systems which, in educational systems till now, have typically been the place where learning activities occur, content is produced, and progress is recorded. By separating the function of recording progress from the means of delivery, it massively increases the range of information and applications that can be used in an integrated way for education, from games to social media systems, MOOCs to traditional classroom activities, and everything between. It allows a great many more forms of progress to be recorded at any level of granularity, from solving a puzzle in a game to looking at a web page, from doing an exam to writing a thesis. It makes it significantly easier to switch between platforms.
  •  it allows any and every kind of evidence of learning to be recorded, of great benefit to lifelong learners wishing to provide evidence of,  and to reflect on and to plan a learning journey. Freeing such data from guarded silos means learners can control their own learning records, keeping them in a cloud, on their own servers or their own computers, transferring them as and when they wish. This goes far beyond the basic functionality of an e-portfolio system, but integrates beautifully with one. It, or something very like it, is a vital piece in the move towards truly open learning at a level that has the potential to disrupt the traditional education and training system.
  •  it makes possible a wide range of analytics for the benefit of learners (and, potentially, for the benefit of teachers and organizations) that go far beyond the structured assessment and activity records that can be captured by an LMS. From finding people with a particular set of skills to analyzing holes in your own learning to organizational learning profiling, the possibilities are huge.

This just scratches the surface of the potential of this technology and standard, and it is just going to get better. More information about Learning Locker is available at http://learninglocker.net

Address of the bookmark: https://github.com/learninglocker/learninglocker

Enough with the lecturing – US National Science Foundation (NSF)

Posted , updated on

Here is a brief report on a somewhat less brief meta-study that claims to show lectures are less effective than active learning approaches in STEM subjects. The report makes a claim that this is an important study. Well, kind of – it will no doubt be cited a lot. This is one of those studies that most educational researchers, including me, would very much like to be correct. Most of our theories and models suggest that lecturing is a truly dumb way to teach. Unfortunately, however, this study does not really show that. We know, and this study confirms in its data, that some lectures work really well some of the time for some students. We also know that active learning approaches do not always work better and sometimes work worse. The study makes the claim that, on average, more active trumps more passive (there are very few blacks and whites in this, it is all on a spectrum, which the study does not very effectively cater for). Sadly, however, it uses a skewed sample because (as this perceptive article highlights very nicely) active approaches typically rely on great, passionate teachers who know how to educate in order to implement them well. And guess who gets to write studies comparing active learning and lecture approaches. Even comparative studies, such as the sort that typically compare a previous run of a course with a newly minted active version, are bound to show this bias. Compare what it is like to teach a course that feels stale with one that is not only new, based on ideas you believe in (which the study, to its credit, attempts to control for) but also (as active learning approaches do) allows your own skill as a teacher to shine. This is systemically biased data.

A message I have been trying to hammer home for some time is that it ain’t what you do, it’s the way that you do it, that’s what gets results. The softer, more malleable, more student-led a course becomes, the more that is required of the skill and passion of the teacher to be responsive and helpful, which means that such courses can be truly awesome or terrifically awful, depending on the people involved. Published research tends to come from those who are more skillful and interested in teaching because those that are not do not normally publish papers about it, so the results available to meta-studies are inevitably skewed.

Lectures are hard, rigid teaching technologies that, together with technologies like textbooks, exams, designed outcomes and timetables, are built to cater for mediocrity in teachers. Great teachers can surpass the limitations an educational system imposes, but some learning happens as a result of process design even when the teacher is truly dire. A dire teacher in an active learning context will leave students confused, lacking direction and demotivated even more than one who just lectures badly.  Furthermore, lectures can have genuine value. I have no great objections in principle to occasional lectures as long as they take up no more than a few minutes of a learner’s day, and as long as lecturers start with the assumption that no direct learning will result from them. Lectures can be good catalysts, a reason to get together, and can help structure studies and thinking, even if they are almost useless for directly learning facts or skills. Just occasionally, not enough to justify their use, they can even inspire and motivate. It ain’t what you do, it’s the way that you do it.

Address of the bookmark: http://nsf.gov/news/news_summ.jsp?cntn_id=131403&org=NSF&from=news

Journal of Interactive Media in Education – Open for Learning Special Issue

Posted , updated on

A special issue of JIME on open learning with 5 chapters (full disclaimer: including one by me and Terry Anderson) from a forthcoming book edited by Chris Pegler and Allison Littlejohn, ‘Reusing Open Resources: Learning in Open Networks for Work, Life and Education’.

I’ve skimmed through the pre-publication draft of the book from which these articles are taken and (not counting our own chapter, about which I may be a little biased) I’m impressed. It has some very important topics, some excellent authors, and a great pair of editors. Deserves to do well.

Terry and I were concerned when responding to the call for chapters about the irony of a book on openness appearing as a closed publication. It is therefore very pleasing that, at least for these five chapters, it is walking the talk. JIME is a fine journal and has been open since it was unfashionable to be so, so I am delighted to at last have an article appear there and congratulate Chris and Allison on a job very well done.

Address of the bookmark: http://www-jime.open.ac.uk/jime/issue/view/2014-ReusingResources-OpenforLearning