Learning in an introductory physics MOOC: All cohorts learn equally, including an on-campus class | Colvin | The International Review of Research in Open and Distance Learning

Thanks to Tony Bates for pointing to and providing a fine review of this interesting article which shows evidence of learning gain in people who were taking an xMOOC.

I have little to add to Tony’s comments apart from to mention the very obvious elephant in this room: that the sampling was skewed by the fact that it only considered considerably less than 10% of the original populace of the MOOC that actually got close to finishing it. It is not too surprising that most of those who had the substantial motivation demanded to finish the course (a large percentage of whom were very experienced learners in related fields) actually did pretty well. What it does not tell us is whether, say, a decent open textbook might have been equally or more effective for these manifestly highly motivated and proficient students. If so, it might not be a particularly cost-effective way of learning.

The study does compare performance of a remedial class of students (ie that had failed an initial course) who received plentiful further face to face support with that of the voluntarily subscribed online students. But the authors rightly note that it would be foolish to read anything into any of the differences found, including the fact that the campus-based students seemed to gain nothing from additional remedial tuition (they may be overly pessimistic about that: without that remedial effort, they might have done even worse) because the demographics and motivations of these students were a million miles removed from the rest of the cohort. Chalk and cheese.

One other interesting thing that is worth highlighting: this is one in a long line of articles focusing on interventions that, when looked at closely, suggest that people who spend more time learning learn more. I suspect that a lot of the value of this and indeed many courses comes from being given permission to learn (or, for the campus students, being made to do so) along with having a few signposts to show the way, a community to learn with, and a schedule to follow. Note that almost none of this has anything to do with how well or how badly a specific course is designed or implemented: it is in the nature of the beast itself. Systems teach as much as teachers. The example of the campus-based students suggests that this may not always be enough although, sadly, the article doesn’t compare the time on task for this group with the rest. It may well be that, despite an extra 4 hours in class each week, they still spent less time actually learning. In fact, given a prima facie case that these students had already mostly demonstrated a lack of interest and/or ability in the subject, then even that tutorial time may have not been dedicated learning time.

A small niggle: the comparison with in-class learning on different courses conducted by Hake in a 1998 study, which is mentioned a couple of times in the article, is quite spurious. There is a world of difference between predominantly extrinsically motivated classroom-bound students and those doing it because, self-evidently, they actually want to do it. If you were to extract the most motivated 10% of any class you might see rather different learning patterns too. The nearest comparison that would make a little sense here is with the remedial campus-bound students though, for aforementioned reasons, that would not be quite fair either.

Little or none of this is news to the researchers, who in their conclusion carefully write:

“Our self-selected online students are interested in learning, considerably older, and generally have many more years of college education than the on-campus freshmen with whom they have been compared. The on-campus students are taking a required course that most have failed to pass in a previous attempt. Moreover, there are more dropouts in the online course (but over 50% of students making a serious attempt at the second weekly test received certificates) and these dropouts may well be students learning less than those who remained. The pre- and posttest analysis is further blurred by the fact that the MOOC students could consult resources before answering, and, in fact, did consult within course resources significantly more during the posttest than in the pretest.”

This is a good and fair account of reasons to be wary of these results. What it boils down to is that there are almost no notable firm conclusions to be drawn from them about MOOCs in general, save that people taking them sometimes learn something or, at least, are able to past tests about them. This is also true of most people that read Wikipedia articles.

For all that, the paper is very well written, the interventions are well-described (and include some useful statistics, like the fact that 95% of the small number that attempted more than 50% of the questions went on to gain a certificate), the research methods are excellent, the analysis is very well conducted, and, in combination with others that I hope will follow, this very good paper should contribute a little to a larger body of future work from which more solid conclusions can be drawn. As Tony says, we need more studies like this.

Address of the bookmark: http://www.irrodl.org/index.php/irrodl/article/view/1902/3009

Political Polarization & Media Habits | Pew Research Center's Journalism Project

The Pew Research Center is responsible for some of the most fascinating and well-conducted research about America and Americans today. In this study, they looked at the relationships between political learnings (conservative vs liberal) and media. It is packed with fascinating details: a lot of the media have picked up on the rather limited range of news channels consumed by those with strong conservative leanings, the polarized trust of many news outlets, and so on. This is not surprising because anything that makes claims about you or your competitors will likely excite interest. But what most fascinates me is the way that social media (Facebook in particular, the generality and ubiquity of which tends to make it a more popular source for news than most other social media, at least in general) contribute to the polarizing effect. Notably, Conservatives tend to see fewer dissenting voices among their feeds. This is not surprising because, though self-reportedly more likely to come across dissenting views,  liberals show a greater tendency to defriend people who express conservative views. The self-organizing network effect caused by this double whammy makes for some dangerous filter bubbles, especially if the main alternative sources of news for American conservatives then appear to be Fox News and Rush Limbaugh.

The trust spectrum is interesting, especially at the extremes. Buzzfeed is trusted by no one, while the Wall Street Journal is trusted by all four of its remaining subscribers. I’d say that it is mighty useful to have a news source that you absolutely do not trust: there’s nothing better to hone your critical faculties. It is most dangerous to trust any media source because it dulls sensibility to stuff and nonsense. At least when you expect limited reliability you are aware of alternative perspectives and the possibility that you are hearing lies, filtered truths and biases.

One of the benefits of old fashioned newspapers, even those with notable biases, is that serendipity always played a role when reading them. Now, with the best of intentions, we get more of the news we explicitly want. When we visit pages, we tend to get recommendations for more of the same (the Landing is ‘guilty’ of this too – we offer recommended content that may be similar whenever you view a page). This is great if you are a learner investigating a topic, not so great if you are hoping to get a well-rounded view of the world. I’m pleased that some people are taking heed of these problems and, rather than reinforcing filter bubbles, they are deliberately bursting them. The Random App (Apple only, sadly) is a good example of a concerted approach to this, mixing random stuff with things that we explicitly express an interest in. We need more of this. It is possible to restore a bit of sane diversity manually: for instance, I get a lot of my news via Pulse, which I have configured with well over 100 feeds, some of which are chosen because they match my interests and leanings, but a lot of which are chosen precisely because they don’t. Crowds are brilliant to learn from if and only if they are sufficiently diverse. 

 

Address of the bookmark: http://www.journalism.org/2014/10/21/political-polarization-media-habits/

Here’s Why Public Wifi is a Public Health Hazard

A nice clear and very graphic explanation of why wifi, especially public wifi, is a very dangerous thing to use. And no, it has nothing whatsoever to do with radiation – if that worries you, and it absolutely shouldn’t, you should be a lot more worried about your TV or radio and positively scared stiff by cellphones, heat lamps and electric stoves. Or light, for that matter. Dangerous stuff, light. 

But, back to the article, most of the more frightening issues it illustrates can be dealt with using a good VPN, use of secure sites (like this one) and very careful attention to what you are clicking and what you are sharing. Others, especially those involving man-in-the-middle attacks and password cracking, can be much trickier to deal with. 

If you are worried by this, and you absolutely should be if any of your devices uses wifi, including your home system, then there are numerous articles that will reassure you that you have some basic safeguards in place, such as: 

  • http://www.forbes.com/sites/amadoudiallo/2014/03/04/hackers-love-public-wi-fi-but-you-can-make-it-safe/ (good basic advice, but does not address some of the issues raised here)
  • http://www.gizmag.com/how-to-stay-secure-on-public-wireless-hotspots/28694/ (a little more complex but a little better informed and offering a little more protection)
  • http://www.watchguard.com/infocenter/editorial/27061.asp (for the geeks or those with a serious interest – a more detailed pair of articles on how wifi evil twins work and what can be done to avoid them, as well as other risks)

If you’ve not thought much about such things, now is a good time.

Address of the bookmark: https://medium.com/matter/heres-why-public-wifi-is-a-public-health-hazard-dd5b8dcb55e6

Dining with an overweight person makes you eat more

It looks like one mechanism for the already observed spread of obesity through social networks may be extremely simple: people tend to eat more when dining with people who are fatter. Thanks to an ingeniously simple experimental design, this paper shows that it’s not due to any difference in the fatter people’s behaviour. It’s solely due to their size. Interesting.

The study deliberately used eating companions for the study, making this a clear network effect in which people are influenced by those with whom they share a reciprocal connection. I’d be intrigued to discover whether it would make any difference if the fatter people (wearing body prostheses) were simply strangers sitting in the same restaurant, not eating together. I’d hypothesise that the effect would still show up, probably more weakly, but that it might be proportional to the number of people who appeared to be obese. In fact, I am guessing it would probably be more complex than that: for instance, that we might be more influenced by those that we thought were more like us or that we took more of a shine to. If so, this would be more of a set than a network effect. It would be not unlike flocking behaviour in birds: until quite recently it was thought that birds flocked due to a simple network effect that spread from neighbour to neighbour but, as it turns out, they are simply counting the birds nearby that are behaving in a particular way, and going with the majority. Memes may work the same way.

This is about as far from intentional communication as it can get – it’s not even a behaviour that is being copied here but some imagined and possibly inaccurate belief about someone’s past behaviour – and yet the effects may be quite profound and, spread through a society, might have massive large scale effects that spread over into many different aspects of many people’s lives, affecting everything from population health to the economy. It’s one of the reasons that schools and universities are a good idea, quite apart from, and independently of, any intentional teaching that might or might not be having an effect. When you see people around you behaving in a particular way, you are more likely to behave similarly. If it seems normal to be actively learning, there’s a much greater chance that you will do so too. Behaviours (even imagined ones) are highly infectious.

Address of the bookmark: http://blogs.discovermagazine.com/seriouslyscience/2014/09/22/dining-overweight-person-makes-others-eat/

StudentLife: Assessing Mental Health, Academic Performance and Behavioral Trends of College Students using Smartphones

A totally fascinating study of students conducted using a massive amount of automatically collected data from smartphones along with other data collected from other systems and via surveys to come up with a large set of correlations relating to everything from mood to GPA. This would win a top paper award in any conference I can think of.

Too much to summarize here, and many more questions emerging from it than it answers, but this should keep a load of researchers busy for years to come. I’m certainly going to be picking this over carefully now that I’ve read it through once. I highly recommend that anyone involved in education (staff or students) should read this! But it should be read with great care and with all critical faculties on full alert. This was a very specific group of students in a very specific context and it would be highly dangerous and irresponsible to extrapolate any generalizations at all from any of this, though I bet some people will. There are lots of things that warrant further investigation – active students were happier and did better but lack of activity, especially at night, seems correlated with higher GPAs, for example, and there are some big fuzzy areas in the sampling that involved a lot of interpretation that was unlikely to be particularly accurate much of the time. The finding that I find particularly appealing is the discovery that classroom attendance had no correlation with academic performance at all: I almost laughed out loud at this one. As always, however it’s not what but how that matters. This suggests to me that someone really needs to work on their classroom activities rather than that classroom teaching does no good, and I would really like to know a lot more about the students who skipped classes before even drawing conclusions from this small dataset let alone more broadly. The other big issues here surround the need for careful interpretation and more qualitative data to explore causes: all this shows are correlations, some of which seem to imply obvious things (e.g. students that study rather than party tend to get better grades but they tend to be lonelier) but many of which are more complex and should be considered in context and at a whole systems level.

The anonymized dataset is available for downloading.

Address of the bookmark: http://studentlife.cs.dartmouth.edu/studentlife.pdf

Professor forces students to buy his own $200 textbook

This article is actually purportedly about the very unsurprising discovery that students who can’t afford textbooks are downloading them illegally, even for ethics classes. Shocking! Not. However, the thing that really shocks me about this article is the example given of the professor demanding that his students purchase his own $200 etextbook. Piracy seems a pretty minor crime compared with this apparently outrageous, blatant, extortionate abuse of power. 

 

Address of the bookmark: http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/09/17/more-students-are-illegally-downloading-college-textbooks-for-free/

The Serious Limitation of Rote Memorisation You Probably Don't Know About (And It's Undermining Learning)

Report on an interesting study showing how rote learning of some things results in increasingly creative interpretations of what we have tried to learn, which means it actually gets in the way of remembering, even though more details are recalled. The researchers note that this is not an issue with simple memorization of numbers, words, etc, but it can be an issue where more complex and relational things need to be recalled – the report mentions understanding the solar system as an example and the researchers used recollection of things in pictures for their study for their testing. In such cases, repetition means more things are remembered, but more things are remembered wrong. I’m wondering whether this affects different kinds of rote memorization, such as the muscle memory used when playing a musical instrument, or learning lines in a song or a play. I’m guessing these are more akin to simple recollections of words because they are a linear sequence, whereas the ways we perceive pictures rely on us choosing where to focus. 

Address of the bookmark: http://www.opencolleges.edu.au/informed/news/limitation-of-rote-learning/

Teaching Crowds: Learning and Social Media

The free PDF preview of the new book by me and Terry Anderson is now available from the AU Press website. It is a complete and unabridged version of the paper book. It’s excellent value!

The book is about both how to teach crowds and how crowds can teach us, particularly at a distance and especially with the aid of social software.

For the sake of your health we do not recommend trying to read the whole thing in PDF format unless you have a very big and high resolution tablet or e-reader, or are unusually comfortable reading from a computer screen, but the PDF file is not a bad way to get a flavour of the thing, skip-read it, and/or to find or copy passages within it. You can also download individual chapters and sections if you wish. 

The paper and epub versions should be available for sale at the end of September, 2014, at a very reasonable price. 

Address of the bookmark: http://www.aupress.ca/index.php/books/120235

Have we all been duped by the Myers-Briggs test?

Not all of us, no.

But, if you reckon there is any validity at all to personality tests, learning styles and all such pseudo-scientific hokum the answer is ‘yes’, you have been duped. This digestible and brief article presents a small sample of the compelling evidence.

Address of the bookmark: http://fortune.com/2013/05/15/have-we-all-been-duped-by-the-myers-briggs-test/

Social Media and the 'Spiral of Silence'

Fascinating but flawed Pew report finding that people are not inclined to share controversial views (at least about Edward Snowden) on social media, and far less so than when talking face-to-face. Moreover, this carries over into face-to-face interactions. The ‘spiral of silence’ in the title is to do with how social media habits become social habits in our physical lives. It’s an evocative phrase that suggests harm is being caused. At least, that is how it is presented and how the press are reporting on it. I’m not convinced. Not at all.

The obvious findings

 

That people are reluctant to stir controversies on social media is not surprising. Most general-purpose social networking systems (which is the subset of social media that the researchers were looking at) indiscriminately connect us and our posts with everyone in our online social network on a given system. This is quite unlike face-to-face converstations where we are intimately aware of our audience and always addressing a smaller subset of the whole, so of course we are aware that some of our audience may not appreciate or care about those opinions. We therefore tend to express them more circumspectly on social media than we do in places where we can better control who we are talking with. This is especially true now that most of us have learned that things are easily misinterpreted online when expressed as text. Moreover, face-to-face, there is little chance that our comments may be happened upon by people some time in the future who are not yet even in our network. This particular problem equally affects social networking systems like the Landing or Google + that make a big point of the ability of members to selectively choose the circles of individuals with whom they share. Given the diversity of the audience and permanence of the result,  it is natural not only to wish not to offend or spark and argument, but also to not want to appear dumb, and to have that dumbness on (semi) permanent record. Face to face, we can correct ourselves when shown to be fools, and no one thinks the less of us but, online, our original foolishness has the same archival permanence as our correction. It is perfectly reasonable to be wary. Furthermore, the the particular issue that Pew chose to investigate (Snowden’s revelations) is far from neutral to the questions they were asking. Thinking about the Snowden/NSA case would likely encourage subjects to focus on the other shadier and malevolent people that might be looking at their posts without their consent or knowledge, not to mention on concerns about privacy on social media in general. Given the nature of the controversy, the survey questions give a reminder that opinions expressed on the subject online might well be used against us. The fact that the study relied upon self-reporting therefore makes it less reliable than it might have been about a less Internet-related topic.

Where it gets interesting

So far so obvious. However, the study goes further. The findings appear to show that social media users are more sensitive to the opinions and beliefs of others, both online and off, and are reluctant to share beliefs that differ from those of social media contacts offline as well as on. This is interesting. The study goes on to say that:

“This suggests a spiral of silence might spill over from online contexts to in-person contexts, though our data cannot definitively demonstrate this causation. It also might mean that the broad awareness social media users have of their networks might make them more hesitant to speak up because they are especially tuned into the opinions of those around them.” 

So, the suggestion is that engagement in social media sensitizes us to others in what might be seen as quite positive ways – social media users care more about what other people think. The downside of this is the fact that we are therefore less likely to challenge things that we see as wrong, less inclined to express dissent, and this may have deep unforeseen consequences for society as a whole. In effect, social media are making us all a little more Canadian! Well, maybe. The researchers laudably note that they cannot demonstrate this causation though, unfortunately and with far less good conscience, they put ‘spiral of silence’ into the title, suggesting that this is what they have actually found and making sensationalist headlines considerably more likely. Bad move.   

There is another interpretation.

I was surprised to discover that “an internet user is 1.63 times more likely to have obtained even a little news on the Snowden-NSA revelations from radio and television than a non-internet user.” So, not only are Internet users more sociable than non-users in real life, they also pay more attention to broadcast sources. This is particularly true of Twitter and Instagram users (3.67 and 4.02 times more likely than a non Internet user, respectively) and less true (though still true) of Pinterest and LinkedIn users.

Though it does discuss possible reasons for inter-media differences, the report does not provide much speculation on the reasons for the internet-user/non-user divide. I speculate (in Devil’s advocate mode) that it might be because it is increasingly hard to find people that are not Internet users, especially in America where this survey was conducted. In this particular case only 456 of 1801 participants were not Internet users, which is surprisingly high at around 25%, but still sufficiently small to make one suspicious. In a good number of cases the reasons for not being Internet-connected are undoubtedly economic, which in itself makes for a demographic that is far from representative. Also, there is probably some significant skewing due to the fact that those who intentionally (whether through fear or deliberate action) divorce themselves from the world of the Internet, even though they could afford it, may have a different attitude to electronic media in general. Either way, this is not comparing like with like. As an overall demographic, non-Internet users are likely becoming increasingly unusual and non-representative of the population as a whole, so perhaps willingness to discuss political hot potatoes is sympomatic of the same thing or things that prevent them from engaging online in the first place. It might well be that what we see on social media has always been the norm and that the skewing in this case is simply revealing that those who do not use the Internet are, on average, different, and they always have been. Maybe they care less what others think or feel. Or maybe their being offline proves that they are more critically atuned. Maybe (and very likely) the topic chosen is one that reinforces and confirms their deep suspicion of the Internet, about which they feel strongly enough both to avoid it and to make their feelings known to others. Whatever, the chances are that there are some common differences.

Equally, the fact that there are also differences between people that use different social media might be due to the kinds of socialization the systems support, rather than differences caused by the media themselves. For instance, most of us choose LinkedIn for professional contacts and information, while we use other sites for different kinds of social activity. People are almost certainly more drawn to social media that support how they tend to want to behave than those that don’t, notwithstanding the strong peer pressure and network effects that make some of them join in anyway. And, just because they join them, doesn’t mean that they use each of them for the same purposes. Those of us that use multiple social media tend to use them differently, with different (often overlapping) networks and different intents. Different levels of political debate are likely not caused by the social media and the networks that inhabit them: we choose those networks because of those differences.

I am not suggesting that any of my speculations are better than the conclusions drawn by the researchers or commentators on the research in the International press (that are, not unexpectedly, very selective about the findings they report on). The point is that this research does not appear to prove my speculations false.  I’m almost sure that the researchers are correct in thinking that different social media reinforce different behaviours, that we are affected by norms and behaviours of others on social media, and that our behaviour in social media has an effect on our behaviour in the physical world. It would be exceedingly odd if it did not. But I’m not at all sure that these results show that. To prove that point I think it would be a minimal requirement to at least do a longitudinal study that shows behavioural changes when people engage in new social media, rather than a snapshot of how they behave now in the context of a topic that is deeply relevant to social media use. It is irresponsible to suggest causation where all that has been found is correlation, with very good grounds for suspecting other causal factors play a significant role and the methodology itself introduces bias. That’s a crucial flaw and it’s a pity, because there is plenty of good information in this report and some thought-provoking findings. It does not need a gutter-press friendly veneer.

Address of the bookmark: http://www.pewinternet.org/files/2014/08/PI_Social-networks-and-debate_082614.pdf