Instructional quality of Massive Open Online Courses (MOOCs)

This is a very interesting, if (I will argue) flawed, paper by Margaryan, Bianco and Littlejohn using a Course Scan instrument to examine the instructional design qualities of 76 randomly selected MOOCs (26 cMOOCs and 50 xMOOCs – the imbalance was caused by difficulties finding suitable cMOOCs). The conclusions drawn are that very few MOOCs, if any, show much evidence of sound instructional design strategies. In fact they are, according to the authors, almost all an instructional designer’s worst nightmare, on at least some dimensions.  
I like this paper but I have some fairly serious concerns with the way this study was conducted, which means a very large pinch of salt is needed when considering its conclusions. The central problem lies in the use of prescriptive criteria to identify ‘good’ instructional design practice, and then using them as quantitative measures of things that are deemed essential to any completed course design. 

Doubtful criteria 

It starts reasonably well. Margaryan et al use David Merrill’s well-accepted abstracted principles for instructional design to identify kinds of activities that should be there in any course and that, being somewhat derived from a variety of models and theories, are pretty reasonable: problem centricity, activation of prior learning, expert demonstration, application and integration. However, the chinks begin to show even here, as it is not always essential that all of these are explicitly contained within a course itself, even though consideration of them may be needed in the design process – for example, in an apprenticeship model, integration might be a natural part of learners’ lives, while in an open ‘by negotiated outcome’ course (e.g. a typical European PhD) the problems may be inherent in the context. But, as a fair approximation of what activities should be in most conventional taught courses, it’s not bad at all, even though it might show some courses as ‘bad’ when they are in fact ‘good’. 
The authors also add five more criteria abstracted from literature relating rather loosely to ‘resources’, including: expert feedback; differentiation (i.e. personalization); collaboration; authentic resources; and use of collective knowledge (i.e. cooperative sharing). These are far more contentious, with the exception of feedback, which almost all would agree should be considered in some form in any learning design (and which is a process thing anyway, not a resource issue). However, even this does not always need to be the expert feedback that the authors demand: automated feedback (which is, to be fair, a kind of ossified expert feedback, at least when done right), peer feedback or, best of all, intrinsic feedback can often be at least as good in most learning contexts. Intrinsic feedback (e.g. when learning to ride a bike, falling off it or succeeding to stay upright) is almost always better than any expert feedback, albeit that it can be enhanced by expert advice. None of the rest of these ‘resources’ criteria are essential to an effective learning design. They can be very useful, for sure, although it depends a great deal on context and how it is done, and there are often many other things that may matter as much or more in a design, like including support for reflection, for example, or scope for caring or passion to be displayed, or design to ensure personal relevance. It is worth noting that Merrill observes that, beyond the areas of broad agreement (which I reckon are somewhat shoehorned to fit), there is much more in other instructional design models that demands further research and that may be equally if not more important than those identified as common.

It ain’t what you do…

Like all things in education, it ain’t what you do but how you do it that makes all the difference, and it is all massively dependent on subject, context, learners and many other things. Prescriptive measures of instructional design quality like these make no sense when applied post-hoc because they ignore all this. They are very reasonable starting frameworks for a designer that encourage focus on things that matter and can make a big difference in the design process, but real life learning designs have to take the entire context into account and can (and often should) be done differently. Learning design (I shudder at the word ‘instructional’ because it implies so many unhealthy assumptions and attitudes) is a creative and situated activity. It makes no more sense to prescribe what kinds of activities and resources should be in a course than it does to prescribe how paintings should be composed. Yes, a few basics like golden ratios, rules of thirds, colour theory, etc can help the novice painter produce something acceptable, but the fact that a painting disobeys these ‘rules’ does not make it a bad painting: sometimes, quite the opposite. Some of the finest teaching I have ever seen or partaken of has used the most appalling instructional design techniques, by any theoretical measure.

Over-rigid assumptions and requirements

One of the biggest troubles with such general-purpose abstractions is that they make some very strong prior assumptions about what a course is going to be like and the context of delivery. Thanks to their closer resemblance to traditional courses (from which it should be clearly noted that the design criteria are derived) this is, to an extent, fair-ish for xMOOCs. But, even in the case of xMOOCs, the demand that collaboration, say, must occur is a step too far: as decades of distance learning research has shown (and Athabasca University proved for decades), great learning can happen without it and, while cooperative sharing is pragmatic and cost-effective, it is not essential in every course. Yes, these things are often a very good idea. No, they are not essential. Terry Anderson’s well-verified (and possibly self-confirming, though none the worse for it) theorem of interaction equivalency  makes this pretty clear.

cMOOCs are not xMOOCs

Prescriptive criteria as a tool for evaluation make no sense whatsoever in a cMOOC context. This is made worse because the traditional model is carried to extremes in this paper, to the extent that the authors bemoan the lack of clear learning outcomes. This doesn’t naturally fall out from the design principles at all, so I don’t understand why they are even mentioned, and it seems an abitrary criterion that has no validity or justification beyond the fact that they are typically used in university teaching. As teacher-prescribed learning outcomes are anathema to Connectivism it is very surprising indeed that the cMOOCs actually scored higher than the xMOOCs on this metric, which makes me wonder whether the means of differentiation were sufficiently rigorous. A MOOC that genuinely followed Connectivist principles would not provide learning outcomes at all: foci and themes, for sure, but not ‘at the end of this course you will be able to x’. And, anyway, as a lot of research and debate has shown, learning outcomes are of far greater value to teachers and instructional designers than they are to learners, for whom they may, if not handled with great care, actually get in the way of effective learning. It’s a process thing – helpful for creating courses, almost useless for taking them. The same problem occurs in the use of course organization in the criteria – cMOOC content is organized bottom-up by learners, so it is not very surprising that they lack careful top-down planning, and that is part of the point.

Apparently, some cMOOCs are not cMOOCs either

As well as concerns about the means of differentiating courses and the metrics used, I am also concerned with how they were applied. It is surprising that there was even a single cMOOC that didn’t incorporate use of ‘collective knowledge’ (the authors’ term for cooperative sharing and knowledge construction) because, without that, it simply isn’t a cMOOC: it’s there in the definition of Connectivism . As for differentiation, part of the point of cMOOCs is that learning happens through the network which, by definition, means people are getting different options or paths, and choosing those that suit their needs. The big point in both cases is that the teacher-designed course does not contain the content in a cMOOC: beyond the process support needed to build and sustain a network, any content that may be provided by the facilitators of such a course is just a catalyst for network formation and a centre around which activity flows and learner-generated content and activity is created. With that in mind it is worth pointing out that problem-centricity in learning design is an expression of teacher control which, again, is anathema to how cMOOCs work. Assuming that a cMOOC succeeds in connecting and mobilizing a network, it is all but certain that a great deal of problem-based and inquiry-based learning will be going on as people post, others respond, and issues become problematized. Moreover, the problems and issues will be relevant and meaningful to learners in ways that no pre-designed course can ever be. The content of a cMOOC is largely learner-generated so of course a problem focus is often simply not there in static materials supplied by people running it. cMOOCs do not tell learners what to do or how to do it, beyond very broad process support which is needed to help those networks to accrete. It would therefore be more than a little weird if they adhered to instructional design principles derived from teacher-led face-to-face courses in their designed content because, if they did, they would not be cMOOCs. Of course, it is perfectly reasonable to criticize cMOOCs as a matter of principle on these grounds: given that (depending on the network) few will know much about learning and how to support it, one of the big problems with connectivist methods is that of getting lost in social space, with insufficient structure or guidance to suit all learning needs, insufficient feedback, inefficient paths and so on. I’d have some sympathy with such an argument, but it is not fair to judge cMOOCs on criteria that their instigators would reject in the first place and that they are actively avoiding. It’s like criticizing cheese for not being chalky enough.

It’s still a good paper though

For all that I find the conclusions of this paper very arguable and the methods highly criticizable, it does provide an interesting portrait of MOOCs using an unconventional lens. We need more research along these lines because, though the conclusions are mostly arguable, what is revealed in the process is a much richer picture of the kinds of things that are and are not happening in MOOCs. These are fine researchers who have told an old story in a new way, and this is enlightening stuff that is worth reading.
 
As an aside, we also need better editors and reviewers for papers like this: little tell-tales like the fact that ‘cMOOC’ gets to be defined as ‘constructivist MOOC’ at one point (I’m sure it’s just a slip of the keyboard as the authors are well aware of what they are writing about) and more typos than you might expect in a published paper suggest that not quite enough effort went into quality control at the editorial end. I note too that this is a closed journal: you’d think that they might offer better value for the money that they cream off for their services.

Address of the bookmark: http://www.sciencedirect.com/science/article/pii/S036013151400178X

7 traits of online graduates that trump campus colleagues

The excellent Donald Clark pinpointing some of the main reasons why it is great to be a graduate of an online university. I’d humbly suggest that Athabasca University is way ahead of the game here: Donald highlights how wonderful it is that these degrees in the UK can be started three whole times a year. We, of course, have a program that starts 12 times a year.

Address of the bookmark: http://donaldclarkplanb.blogspot.ca/2014/11/7-traits-of-online-graduates-that-trump.html

Microsoft Open Sources .NET, Saying It Will Run On Linux and Mac | WIRED

This is a sign of what appear to be some remarkable seismic shifts at Microsoft. To be fair, Microsoft has long been a contributor to open source initiatives but .NET was, until fairly recently, seen as one of the crown jewels only slightly less significant than Windows and Office, which makes me and the writer of this article wonder whether they might be heading towards open sourcing these at some point (Windows mobile version is already free, albeit with many provisos, terms and conditions, but that’s just common sense otherwise no one would use the substandard pile of pants at all).

Note that they are apparently only open-sourcing the core of .NET, which is not that wonderful without all the accompanying framework and goodies. The open source Mono project has provided this functionality for many years thanks to Microsoft’s wisely open approach to treating it and C# as a specification rather than a completely closed technology in the first place but, and it’s a big but, there are few Windows .NET apps that can run on Mono under Unix without some significant tweaking or acceptance of limitations and bugs, because so much relies on the premium libraries, controls and other proprietary closed tools that only paying Windows users can take advantage of. It’s much better than it used to be, but Mono is still a shim rather than a solution. I’m guessing there are few that would use it in preference to, say, Java unless their primary target were Windows machines or they were inveterate C# or VB fans.

This is probably not a sign of deeper openness, however. Microsoft, like most others in the industry, clearly see the future is in net-delivered cloud-based subscription services. Azure, Office365, Skype, Exchange Online etc etc are likely to be where most of the money comes from in the years ahead. .NET is nothing like as effective at locking people in than providing a service that handles all the data, communication and business processes of an individual or organization. Moreover, if more .NET developers can be sucked in to developing for other platforms, that means more that can be pulled in to Microsoft’s cloud systems though, to be fair, it does mean Microsoft has to actually compete on even ground to win, rather than solely relying on market dominance. But it does have a lot of cash to outspend many of its rivals, and raw computer power together with the money to support it plays a large role in achieving success in this area.

The cloud is a new (well, extremely old but now accepted and dominant) form of closed system in which the development technology really shouldn’t matter much any more. I worry a great deal about this though. In the past we were just locked in by data formats, closed licences and closed software (perniciously driven by upgrade cycles that rendered what we had purchased obsolete and unsupported), but at least the data were under our control. Now they are not. I know of no cloud-based services that have not at some point changed terms and conditions, often for the worse, few that I would trust with my data any further than I could throw them, and none at all that are impervious to bankrupcy, take-overs and mergers. When this happened in the past we always had a little lead time to look for an alternative solution and our systems kept running. Nowadays, a business can be destroyed in the seconds it takes to shut down or alter a system in the cloud.

Address of the bookmark: http://www.wired.com/2014/11/microsoft-open-sources-net-says-will-run-linux-mac/

Multiple types of motives don't multiply the motivation of West Point cadets

Interesting study analysing the relationship between internal vs instrumental (the author’s take on intrinsic vs extrinsic) motivation as revealed in entry questionnaires for West Point cadets and long-term success in army careers. As you might expect, those with intrinsic motivation significantly outperformed those with extrinsic motivation on every measure.

What is particularly interesting, however, is that extrinsic motivation crowded out the intrinsic in those with mixed motivations. Having both extrinsic and intrinsic motivation is no better than having extrinsic motivation on its own, which is to say it is virtually useless. In other words, as we already know from hundreds of experiments and studies over shorter periods but herein demonstrated for periods of over a decade, extrinsic motivation kills intrinsic motivation. This is further proof that the use of rewards (like grades, performance-related pay, and service awards) in the hope that they will motivate people is an incredibly dumb idea because they actively demotivate. 

Address of the bookmark: http://m.pnas.org/content/111/30/10990.full

Zombie Skinner returns from the dead: an educational horror story

A dark tale for the halloween season

It was a dark and stormy night when Phil deMuth, an investment advisor, sat down to pen this article for Forbes. His voodoo incantations would raise from the dead the ghastly zombified remains of behaviourist dogmatist and appropriately named Skinner and, together, their nightmarish vision would turn the nation’s young into mindless zombies: apathetic, disenfranchised, undead and unthinking fodder suited only to sustaining the ghost of an industrial past. It is because of idiotic but superficially plausible ideas like the ones in this skillfully written article that I have to struggle to unteach, to try and often fail to help students to learn how to love learning again after years of having it beaten out of them. So, in case anyone is persuaded by the slick but outrageously wrong arguments of the article, or is one of the many educators that actually make use of this claptrap, this post is meant as a small antidote to the zombie plague.

A good beginning

DeMuth’s article starts very well. The first six paragraphs of the article present a fine and impassioned analysis of the failings of popular educational technologies and methods that strikes a well-aimed blow at the heart of many of the problems in existing educational systems, the atrociousness of traditional methods, and the unwitting replication of harmful and outmoded ways of teaching in poorly designed MOOCs and misuse of Khan tutorials (he actually attacks Khan tutorials themselves but, had he thought to check out a few rather than lump them with the rest, he would have discovered that they actually closely conform to his vision). deMuth might also have attacked much of university e-learning too on the same grounds. I could not agree more. DeMuth is absolutely right to bemoan the crazy transmission model underlying much of education and the appallingness of monolithic and intimidating exams and tests, not to mention the foolishness of replicating old and weak methods in new and shiny tools. His motives are unimpeachable and he makes a very strong, eloquently argued case. His solution though is not to change that system but to make it do the (wrong) job more efficiently. Here I take issue.

And then the horror starts

The remainder of the article is entertainingly and slickly written but the worse for that, because it carries a very dangerous message indeed.  In brief, it spends a while self-referentially demonstrating the value of programmed learning, then winds up by asking for programmers/behaviourist psychologists to produce modern equivalents of Skinner’s Teaching Machine (see illustration) so that the transmission model can work better and kids can pass more tests.

Behaviourism revisited

B.F.Skinner's Teaching Machine

I’m amazed that anyone still thinks radical behaviourism, as espoused by Skinner, has any value whatsoever. I guess that some people learned this stuff before it was soundly discredited or, as I’m guessing was the case for deMuth, discovered it in passing without looking into what the rest of the world thought about it. Alas, such beliefs still do still persist. Indeed, we see behaviourist shortcuts all too regularly in education and industry to this day, even though hardly anyone who has followed any of the research over the past 40 years or so would find it at all acceptable. 

For those unfamiliar with Skinner’s radical behaviourist model, in brief, it was meant to apply a reductionist scientific method to discovering how animals (including people) learn. Recognizing that internal cognitive processes are hard to observe (and, in Skinner’s radical version of the theory, are themselves simply a consequence of external conditioning), the ‘behaviour’ part of the name is a reflection of that fact that this is what behaviourists concentrated on and, in Skinner’s case, the only thing that counted. Skinner only allowed for stimuli and responses that could be observed and measured – nothing else mattered. The brain for Skinnerian behaviourists was a black box about which they needed to know nothing apart from the effects of particular inputs and the observable outputs. They performed interventions and observed their effects on behaviour. Based on these observations (not uncommonly starting and sometimes ending with experiments on animals) those who tried to apply these methods in teaching sought to work out how to teach better, without ever having to make any assumptions about what was going on inside people’s minds. It’s a laudable goal, if Quixotic and utterly misguided. One big trouble with it (though far from the only one) is that it ignores our minds’ own inputs, that are often a great deal more significant than any external stimuli and that always modify them, in unpredictable ways, with complex effects that often fail to emerge until long after the stimuli have gone. We now know that reductionist methods simply don’t work in this context. Skinner lacked the framework of complexity and chaos theories that demonstrate the theoretical and practical impossibility of predicting even such simple causes and effects as the motion of a double pendulum, so it is perhaps a little forgiveable that he remained lost in a reductionist paradigm. We also now know that the operant conditioning methods that Skinner espoused are relatively ineffective in the short term and highly ineffective in the long term, so behaviourism fails to achieve much even on its own terms. Again, Skinner could not really be blamed for misunderstanding the significance of his results or their long-term weaknesses because such research was in its infancy while he was still alive.

So why did anyone ever believe in this stuff?

To a limited extent, behaviourism works. Among radical behaviourist ‘discoveries’, in large part guided by experiments in which Skinner was able to train animals like pigeons little by little to perform complex tasks, is the one focused on in this article: that small chunked lessons, with immediate feedback, allowing the subject to take it at his or her own pace, can reliably lead to learning. Up to a point this is absolutely true, especially in the ‘spaced’ form in which Skinner actually presented it rather than the simplistic caricature demonstrated in the article. For some kinds of rote learning, the effects of which are easily observed, small chunks and immediate feedback are a very good idea indeed. There are good reasons for this that cognitivist psychologists and constructivist thinkers had also hit upon long before Skinner came on the scene. Although not the archetypal behaviourist way of doing things, it works particularly well if that feedback is innate to the task rather than extrinsically imposed. For instance, staying upright on a bicycle, being able to recite lines in a play, being able to play a piece of music, building a program that does what it should, or writing a satisfying piece of work, are immediate forms of feedback that are intrinsic to the process. This is the form that deMuth uses in the article: he self-referentially demonstrates the effectiveness of the approach by leaving ever larger chunks out of the key terms it employs. In doing this he is actually relying on a cognitivist model of what motivates us (in this case, achievable challenges) rather than a purely behaviourist model of reward and punishment, so it’s not the greatest example of the effectiveness of behaviourism. He is not exactly an expert in the field. Behaviourists also hit on a few other good tricks, more by luck than design. It is absolutely true that putting people in control of the pace of their own learning works very well, both for obvious common-sense reasons (we don’t all learn the same things at the same speed) and for motivation: a sense of being in control is central to intrinsic motivation. This is not a behaviourist notion, but it happens to be true.

The most problematic outcome of behaviourist thinking, that follows from its wilful ignorance of internal motivations and stimuli, lies in the use of rewards and punishments to drive learning, using extrinsic motivation as though we were all pigeons. The big trouble with the reward/punishment idea is that extrinsic motivation actually eliminates intrinsic motivation, which means that a reward/punishment model is positively harmful to effective learning. There have been countless studies and experiments that show this from Deci, Ryan, Kohn and very many others. I am quite taken by a recent paper on the subject which rather neatly shows its effects using 10,000 West Point cadets tracked over 14 years, that summarizes some of the classic research quite well, as well as adding its own compelling evidence. By nature humans love to learn and enjoy achievable challenges but, if you beat or reward that love out of them for long enough, they will stop wanting to do so. The big lesson that we learn from extrinsic rewards and punishments is that learning is done to gain rewards that have no connection with that learning, or (just as bad) to avoid punishment. We also, in passing, learn that the purveyors of those rewards and punishments have power over us.

If it actually worked then even this ugly power trip might be worth it, though I have strong reservations about the ultimate value of teaching people to bow down to authority figures without question. Unfortunately, many studies have demonstrated unequivocally that, though such methods may result in short-term gains that may be sufficient (if not particularly efficient) to pass the big sticks and carrots of exams designed to test behaviour, learning this way does not persist, especially if no attention is paid to meaning, value and connections between things. This accords with common sense and experience. If we are taught that the value of what we are learning is to pass a test or get a grade then, once we have achieved that, it is perfectly natural to promptly forget it. It’s much like remembering your hotel room number: very important as long as you stay there, completely irrelevant when you leave, and therefore promptly forgotten.

Learning that persists is learning that we can continue to use, that relates to our goals, the things we want or need to do, that relates to our social context, that we can apply and that has meaning and value to us because of who we are, where we come from, what we want to do, the communities that bind us, and who we want to be. For this kind of learning, self-pacing, small chunks of increasing complexity and fast feedback can be extremely useful tools (if far from being the only ones) but the point is that it cannot be done effectively in isolation and especially not under the control of someone else. Values and meaning are not in nor can they be usefully described by the behaviourist vocabulary, but they are exactly what deMuth’s reviled educators, against the odds and against the flow of a system that is designed to work in total opposition to them, are trying to foster. At least, the good ones are doing that. Too many of us are buckled down by the system that thwarts us by standardizing learning, trying to make us teach the same things in the same temporal and physical/virtual space, at the same time, over fixed periods, without any thought for the reasons it might be worthwhile to people or their individual and unique needs. It is no wonder that education has one of the highest dropout rates of any profession. DeMuth rightly attacks education but wrongly attacks educators.

The failure of educational systems

As long as we have abominations like the core curriculum, obligatory courses with defined objectives, or coarse-grained programs that ignore individual needs, that make it a requirement to learn a specified body of facts and skills regardless of their personal value or interest to us, this will ever be so. Unless we can devise ways of doing education that will be meaningful, applicable and valuable to the individuals that are learning, without extrinsic rewards or punishments, we have failed. If we teach students that the purpose of learning is to pass tests (or receive some other extrinsic reward or punishment), we will have doubly failed, because we will have made it harder for them to learn anything ever again and, in all probability, will have fostered an aversion to what might otherwise have been an important and interesting thing if it were learned at the right time. Skinnerian teaching machines deployed without addressing these fundamental problems will simply reinforce the same old patterns, making things worse, far worse, than before.

When used to support personally and socially meaningful goals, some behaviourist methods can have limited value, though none of those methods are unique to behaviourism and most come with important provisos and modifiers. Practice can be very good for acquiring a wide range of skills, especially when interleaved and spaced (those studying for exams or learning to play a musical instrument would do well to take note of this), learning things when, how and at what pace we wish to learn them is crucial, and we do need to take things a little at a time. Some such skills are foundational and, once learned, can become self-sustaining and supportive of intrinsically motivated learning: reading and writing, for example, or arithmetic. But behaviourism is not right just because it made a few hits. DeMuth wants to improve literacy (good) but he seeks to improve it through behaviourist methods (very bad) and measure it by standardized tests (very, very bad). This is a bit like assuming that the purpose of the army is to kill people, and therefore providing all soldiers with nuclear weapons. It is putting the cart before the horse. 

The purpose of education is not to pass tests but, along with sustaining some cultural continuity, to help people both to learn and to continue to learn. Behaviourist methods may achieve short-term testing goals but are singularly poor at fostering long-term learning and are positively antagonistic to lifelong learning. They encourage a dependent and submissive attitude and stamp on critical or creative thought. We should let them rest in peace.

BOOK: Teaching Crowds: Learning and Social Media

About the Book

Within the rapidly expanding field of educational technology, learners and educators must confront a seemingly overwhelming selection of tools designed to deliver and facilitate both online and blended learning. Many of these tools assume that learning is configured and delivered in closed contexts, through learning management systems (LMS). However, while traditional “classroom” learning is by no means obsolete, networked learning is in the ascendant. A foundational method in online and blended education, as well as the most common means of informal and self-directed learning, networked learning is rapidly becoming the dominant mode of teaching as well as learning.

In Teaching Crowds, Dron and Anderson introduce a new model for understanding and exploiting the pedagogical potential of Web-based technologies, one that rests on connections — on networks and collectives — rather than on separations. Recognizing that online learning both demands and affords new models of teaching and learning, the authors show how learners can engage with social media platforms to create an unbounded field of emergent connections. These connections empower learners, allowing them to draw from one another’s expertise to formulate and fulfill their own educational goals. In an increasingly networked world, developing such skills will, they argue, better prepare students to become self-directed, lifelong learners.

 

Address of the bookmark: http://www.aupress.ca/index.php/books/120235

Transactional distance and new media literacies

Moore’s theory of transactional distance describes the communications and psychological gulf between learner and teacher in a distance education setting. The theory was formulated in a correspondence era of distance learning and matured in an era where discussion forums and virtual learning environments reduced transactional distance in a closed-group setting that enabled interactions akin to those in a traditional classroom. In recent years the growth of social networking and social interest sites has led to social forms that fit less easily in these traditional formal models of teaching and learning. When the “teacher” is distributed through the network or is an anonymous agent in a set or is an emergent actor formed by collective intelligence, transactional distance becomes a more complex variable. Evolved social literacies are mutated by new social forms and require us to establish new or modified ways of thinking about learning and teaching. In this missive we explore the notion of transactional distance and the kinds of social literacy that are required for or that emerge from network, set, and collective modes of social engagement. We discuss issues such as preferential attachment, confirmation bias, and trust and describe social literacies needed to cope with them.

Address of the bookmark: http://www.mitpressjournals.org/doi/abs/10.1162/IJLM_a_00104#.VEwtAYcfTEI

Agoraphobia and the modern learner

Abstract:Read/write social technologies enable rich pedagogies that centre on sharing and constructing content but have two notable weaknesses. Firstly, beyond the safe, nurturing environment of closed groups, students participating in more or less public network- or set-oriented communities may be insecure in their knowledge and skills, leading to resistance to disclosure. Secondly, it is hard to know who and what to trust in an open environment where others may be equally unskilled or, sometimes, malevolent. We present partial solutions to these problems through the use of collective intelligence, discretionary disclosure controls and mindful design.

Address of the bookmark: http://www-jime.open.ac.uk/jime/article/viewArticle/2014-03/html

Learning in an introductory physics MOOC: All cohorts learn equally, including an on-campus class | Colvin | The International Review of Research in Open and Distance Learning

Thanks to Tony Bates for pointing to and providing a fine review of this interesting article which shows evidence of learning gain in people who were taking an xMOOC.

I have little to add to Tony’s comments apart from to mention the very obvious elephant in this room: that the sampling was skewed by the fact that it only considered considerably less than 10% of the original populace of the MOOC that actually got close to finishing it. It is not too surprising that most of those who had the substantial motivation demanded to finish the course (a large percentage of whom were very experienced learners in related fields) actually did pretty well. What it does not tell us is whether, say, a decent open textbook might have been equally or more effective for these manifestly highly motivated and proficient students. If so, it might not be a particularly cost-effective way of learning.

The study does compare performance of a remedial class of students (ie that had failed an initial course) who received plentiful further face to face support with that of the voluntarily subscribed online students. But the authors rightly note that it would be foolish to read anything into any of the differences found, including the fact that the campus-based students seemed to gain nothing from additional remedial tuition (they may be overly pessimistic about that: without that remedial effort, they might have done even worse) because the demographics and motivations of these students were a million miles removed from the rest of the cohort. Chalk and cheese.

One other interesting thing that is worth highlighting: this is one in a long line of articles focusing on interventions that, when looked at closely, suggest that people who spend more time learning learn more. I suspect that a lot of the value of this and indeed many courses comes from being given permission to learn (or, for the campus students, being made to do so) along with having a few signposts to show the way, a community to learn with, and a schedule to follow. Note that almost none of this has anything to do with how well or how badly a specific course is designed or implemented: it is in the nature of the beast itself. Systems teach as much as teachers. The example of the campus-based students suggests that this may not always be enough although, sadly, the article doesn’t compare the time on task for this group with the rest. It may well be that, despite an extra 4 hours in class each week, they still spent less time actually learning. In fact, given a prima facie case that these students had already mostly demonstrated a lack of interest and/or ability in the subject, then even that tutorial time may have not been dedicated learning time.

A small niggle: the comparison with in-class learning on different courses conducted by Hake in a 1998 study, which is mentioned a couple of times in the article, is quite spurious. There is a world of difference between predominantly extrinsically motivated classroom-bound students and those doing it because, self-evidently, they actually want to do it. If you were to extract the most motivated 10% of any class you might see rather different learning patterns too. The nearest comparison that would make a little sense here is with the remedial campus-bound students though, for aforementioned reasons, that would not be quite fair either.

Little or none of this is news to the researchers, who in their conclusion carefully write:

“Our self-selected online students are interested in learning, considerably older, and generally have many more years of college education than the on-campus freshmen with whom they have been compared. The on-campus students are taking a required course that most have failed to pass in a previous attempt. Moreover, there are more dropouts in the online course (but over 50% of students making a serious attempt at the second weekly test received certificates) and these dropouts may well be students learning less than those who remained. The pre- and posttest analysis is further blurred by the fact that the MOOC students could consult resources before answering, and, in fact, did consult within course resources significantly more during the posttest than in the pretest.”

This is a good and fair account of reasons to be wary of these results. What it boils down to is that there are almost no notable firm conclusions to be drawn from them about MOOCs in general, save that people taking them sometimes learn something or, at least, are able to past tests about them. This is also true of most people that read Wikipedia articles.

For all that, the paper is very well written, the interventions are well-described (and include some useful statistics, like the fact that 95% of the small number that attempted more than 50% of the questions went on to gain a certificate), the research methods are excellent, the analysis is very well conducted, and, in combination with others that I hope will follow, this very good paper should contribute a little to a larger body of future work from which more solid conclusions can be drawn. As Tony says, we need more studies like this.

Address of the bookmark: http://www.irrodl.org/index.php/irrodl/article/view/1902/3009

Political Polarization & Media Habits | Pew Research Center's Journalism Project

The Pew Research Center is responsible for some of the most fascinating and well-conducted research about America and Americans today. In this study, they looked at the relationships between political learnings (conservative vs liberal) and media. It is packed with fascinating details: a lot of the media have picked up on the rather limited range of news channels consumed by those with strong conservative leanings, the polarized trust of many news outlets, and so on. This is not surprising because anything that makes claims about you or your competitors will likely excite interest. But what most fascinates me is the way that social media (Facebook in particular, the generality and ubiquity of which tends to make it a more popular source for news than most other social media, at least in general) contribute to the polarizing effect. Notably, Conservatives tend to see fewer dissenting voices among their feeds. This is not surprising because, though self-reportedly more likely to come across dissenting views,  liberals show a greater tendency to defriend people who express conservative views. The self-organizing network effect caused by this double whammy makes for some dangerous filter bubbles, especially if the main alternative sources of news for American conservatives then appear to be Fox News and Rush Limbaugh.

The trust spectrum is interesting, especially at the extremes. Buzzfeed is trusted by no one, while the Wall Street Journal is trusted by all four of its remaining subscribers. I’d say that it is mighty useful to have a news source that you absolutely do not trust: there’s nothing better to hone your critical faculties. It is most dangerous to trust any media source because it dulls sensibility to stuff and nonsense. At least when you expect limited reliability you are aware of alternative perspectives and the possibility that you are hearing lies, filtered truths and biases.

One of the benefits of old fashioned newspapers, even those with notable biases, is that serendipity always played a role when reading them. Now, with the best of intentions, we get more of the news we explicitly want. When we visit pages, we tend to get recommendations for more of the same (the Landing is ‘guilty’ of this too – we offer recommended content that may be similar whenever you view a page). This is great if you are a learner investigating a topic, not so great if you are hoping to get a well-rounded view of the world. I’m pleased that some people are taking heed of these problems and, rather than reinforcing filter bubbles, they are deliberately bursting them. The Random App (Apple only, sadly) is a good example of a concerted approach to this, mixing random stuff with things that we explicitly express an interest in. We need more of this. It is possible to restore a bit of sane diversity manually: for instance, I get a lot of my news via Pulse, which I have configured with well over 100 feeds, some of which are chosen because they match my interests and leanings, but a lot of which are chosen precisely because they don’t. Crowds are brilliant to learn from if and only if they are sufficiently diverse. 

 

Address of the bookmark: http://www.journalism.org/2014/10/21/political-polarization-media-habits/