Wisdom of the Confident: Using Social Interactions to Eliminate the Bias in Wisdom of the Crowds

A really interesting paper on making crowds smarter.  I find the word ‘confident’ in the title a bit odd because it seems (and I may have misunderstood) that the researchers are actually trying to measure independent thinking rather than confidence. As far as I can tell, this describes a method for separating sheep (those more influenced by others) from goats (those making more independent decisions), at least when you have a sequence of decisions/judgments to work with. The reason it bothers me is that sheep can be confident too (see the US election or Brexit, for example).

We know that crowds can be wise if and only if the agents in the crowd are unaware of the decisions of other agents. If there’s a feedback loop (more accurately, I believe, if there is an insufficiently delayed feedback loop) then you wind up with stupid mobs, driven by preferential attachment and similar dynamics. This is a big problem in many political systems that allow publication of polls and early results. However, some people are, for one reason or another, less influenced by the crowd than others. It would be useful to be able to aggregate their decisions while ignoring those that simply follow the rest, in order to achieve wiser crowds. That’s what the method described here seeks to do.

The paper is more concerned with describing its model than with describing or analyzing the experiment itself, which is a pity as I’d like to know more about the populations used and tasks performed, and whether it really is discriminating confident from independent behaviour. I’ve also done some work in this area and have written about how useful it would be to automatically identify independent thinkers, and to use their captured behaviour instead of that of the whole crowd to make decisions, but I have never implemented that because, in real life, this is quite hard to do. In this experiment, it seems quite possible that the ‘independent’ people might simply have been those that knew more about the domain. That’s great if we are using a sequence of captured data from the same domain (in this case, length of country borders) because we get results from those that know rather than those that guess. But it won’t transfer when the domain changes even slightly: knowing the length of the Swiss border might not well predict knowledge of, say, the length of the Nigerian border, though I guess it might improve things slightly because those that care about such things would be better represented in the sample.

It would take a fair bit of evidence, I suspect, to identify someone as a context-independent independent thinker though, given enough time, it could be done, it would be well worth doing, and this model might provide the means to identify that. I’d like to see it applied in a real context. There are less lengthy and privacy-invading alternatives. For instance, we might capture both a rating/value/judgement/whatever and some measure of confidence. Some kinds of prediction market capture that sort of data and, because of the personal stake in it, might achieve better results when we do not have a long history of data to analyze. Whether and to what extent confidence is related to independence, and whether the results would be better remains to be discovered, of course – there’s a good little research project to be done here – but it would be a good start.

Address of the bookmark: https://arxiv.org/abs/1406.7578

Setapp – Netflix-style rental model for apps for Mac

Interesting. For $10USD/month, you get unlimited access to the latest versions of what is promised to be around 300 commercial Mac apps. Looking at the selection so far (about 50 apps), these appear to be of the sort that usually appear in popular app bundles (e.g. StackSocial etc), in which you can buy apps outright for a tiny fraction of the list price (quite often at a 99% reduction). I have a few of these already, for which I paid an average of 1 or 2 dollars apiece, albeit that they came with a bunch of useless junk that I did not need or already owned, so perhaps it’s more realistic to say they average more like $10 apiece. Either way, they can already be purchased for very little money, if you have the patience to wait for the right bundle to arrive. So why bother with this?

The main advantage of SetApp’s model is that, unlike those in bundles, which often nag you to upgrade to the next version at a far higher price than you paid almost as soon as you get them, you always get the latest version. It is also nice to have on-demand access to a whole library at any time: if you can wait for a few months they will probably turn up in a cheap pay-what-you-want app bundle anyway, but they are only rarely available when you actually need them.  I guess there is a small advantage in the curation service, but there are plenty of much better and less inherently biased ways to discover tools that are worth having. 

The very notable disadvantage is that you never actually own the apps – once you stop subscribing or the company changes conditions/goes bust, you lose access to them. For ephemerally useful things like disk utilities, conversion tools, etc this is no great hassle but, for things that save files in proprietary formats or supply a cloud service (many of them) this would be a massive pain. As there is (presumably) some mechanism for updating and checking licences, this might also be an even more massive pain if you happen to be on a plane or out of network range when either the app checks in or the licence is renewed. I don’t know which method SetApp uses to ensure that you have a subscription but, one way or another, lack of network access at some point in the proceedings could really screw things up. When (with high probability) SetApp goes bust, you will be left high and dry. Also, I’m guessing that it is unlikely that I would want more than a dozen or thereabouts of these in any given year, so each would cost me about $10 every year at the best of times. Though that might be acceptable for a major bit of software on which one’s livelihood depends, for the kind of software that is currently on show, that’s quite a lot of money, notwithstanding the convenience of being able to pick up a specialist tool when you need it at no extra cost. 

This is a fairly extreme assault on software ownership but closed-source software of all varieties suffers from the same basic problem: you don’t own the software that you buy.  Unlike use-once objects like movies or books, software tends to be of continuing value. The obvious solution is to avoid closed-source altogether and go for open source right the way down the stack: that’s always my preference. Unfortunately, there are still commercial apps that I find useful enough to pay for and, unfortunately, software decays. Even if you buy something outright that does the job perfectly, at some point the surrounding ecosystems (the operating system, network, net services, etc) will most likely render it useless or positively dangerous at some point. There are also some doubly annoying cases where companies stop supporting versions, lose databases, or get taken over by other companies, so software that you once owned and paid for is suddenly no longer yours (Cyberduck, I’m looking at you). Worst of all are those that depend on a cloud service over which you have no control at all and that will almost definitely go bust, or get taken over, or be subject to cyberattack, or government privacy breaches, or be unavailable when you need it, or that will change terms and conditions at some point to your extreme disadantage. Though there may be a small niche for such things and the immediate costs are often low enough to be tempting, as a mainstream approach to software provision, it is totally unsustainable.

 

Address of the bookmark: https://setapp.com/

Pebble dashed

Hell.

Pebble made my favourite smart watches. They were somewhat open, and the company understood the nature of the technology better than any of the mainstream alternatives. Well, at least they used to get it, until they started moving towards turning them into glorified fitness trackers, which is probably why the company is now being purchased by Fitbit.

So, no more Pebble and, worse, no more support for those that own (or, technically, paid for the right to use) a Pebble. If it were an old fashioned watch I’d grumble a bit about reneging on warranties but it would not prevent me from being able to use it. Thanks to the cloud service model, the watch will eventually stop working at all:

Active Pebble watches will work normally for now. Functionality or service quality may be reduced down the road. We don’t expect to release regular software updates or new Pebble features. “

Great. The most expensive watch I have ever owned has a shelf life of months, after which it will likely not even tell the time any more (this has already occurred on several occasions when it has crashed while I have not been on a viable network). On the bright side (though note the lack of promises):

We’re also working to reduce Pebble’s reliance on cloud services, letting all Pebble models stay active long into the future.”

Given that nearly all the core Pebble software is already open source, I hope that this means they will open source the whole thing. This could make it better than it has ever been. Interesting – the value of the watch would be far greater without the cloud service on which it currently relies. 

 

Address of the bookmark: https://www.kickstarter.com/projects/597507018/pebble-2-time-2-and-core-an-entirely-new-3g-ultra/posts/1752929

The cost of admission to the unlearning zone

picture of dull classroom (pubic domain)I describe some of what I do as ‘unteaching’, so I find this highly critical article by Miss Smith – The Unlearning Zone –  interesting. Miss Smith dislikes the terms ‘ unteaching’ and ‘unlearning’ for some well-expressed aesthetic and practical reasons: as she puts it, they are terms “that would not be out of place in a particularly self-satisfied piece of poststructuralist literary analysis circa 1994.”  I partially agree. However, she also seems equally unenamoured with what she thinks they stand for. I disagree with her profoundly on this so, as she claims to be new to these terms, here is my attempt to explain a little about what I mean by them and why I think they are a useful part of the educators’ lexicon, and why they are crucially important for learners’ development in general.

First the terms…

Yes, ‘unteaching’ is an ugly neoligism and it doesn’t really make sense: that’s part of the appeal of using it – a bit of cognitive dissonance can be useful for drawing attention to something. However, it is totally true that someone who is untaught is just someone who has not (yet) been taught, so ‘unteaching’, seen through that light, is at best pointless, at worst self-contradictory.  On the other hand, it does seem to follow pretty naturally from ‘unlearning’ which, contrary to Miss Smith’s assertion, has been in common use for centuries and makes perfect sense. Have you ever had to unlearn bad habits? Me too.

As I understand it, ‘unteach’ is to ‘teach’ as ‘undo’ is to ‘do’.  Unteaching is still teaching, just as undoing is still doing, and unlearning is still learning. Perhaps deteaching would be a better term. Whatever we choose to call it, unteaching is concerned with intentionally dismantling the taught belief that teaching is about exerting power over learners to teach, and replacing it with the attitude that teachers are there to empower learners to learn. This is not a particularly radical idea. It is what all teachers should do anyway, I reckon. But it is worth drawing attention to it as a distinct activity because it runs counter to the tide, and the problem it addresses is virtually ubiquitous in education up to, and sometimes at, doctoral level.

Traditional teaching of the sort Miss Smith seems to defend in her critique does a lot more than teach a subject, skill, or way of thinking. It teaches that learning is a chore that is not valuable in and of itself, that learners must be forced to do it for some other purpose, often someone else’s purpose. It teaches that teaching is something done to students by a teacher: at its worst, it teaches that teaching is telling; at best, that teaching involves telling someone to do something. It’s not that (many) teachers deliberately seek these outcomes, but that they are the most likely lessons to be learned, because they are the ones that are repeated most often. The need for unteaching arises because traditional teaching, with luck in addition to whatever it intends to teach, teaches some terrible lessons about learning and the role of teaching in that process that must be unlearned.

What is unteaching?

Miss Smith claims that unteaching means “open plan classes, unstructured lessons and bean bags.” That’s not the way I see it at all. Unlike traditional teaching, with its timetables, lesson plans, learning objectives, and uniform tests, unteaching does not have its own technologies and methods, though it does, for sure, tend to be a precursor to connectivist, social constructivist, constructionist, and other more learner-centred ways of thinking about the learning process, which may sometimes be used as part of the process of unteaching itself. Such methods, models, and attitudes emerge fairly naturally when you stop forcing people to do your bidding. However, they are just as capable of being used in a controlling way as the worst of instructivist methods: the number of reports on such interventions that include words like ‘students must…’, ‘I make my students…’ or (less blatantly) ‘students (do X)’ far outnumber all others, and that is the very opposite of unteaching. The specific technologies (including pedagogies as much as open-plan classrooms and beanbags) are not the point. Lectures, drill-and-practice and other instructivist methods are absolutely fine, as long as:

  1. they at least attempt to do the job that students want or need,
  2. they are willingly and deliberately chosen by students,
  3. students are well-informed enough to make those choices, and
  4. students can choose to learn otherwise at any time.

No matter how cool and groovy your problem-based, inquiry-based, active methods might be, if they are imposed on students (especially with the use of threats for non-compliance and rewards for compliance – e.g. qualifications, grades, etc) then it is not unteaching at all: it’s just another way of doing the same kind of teaching that caused the problem in the first place. But if students have control – and ‘control’ includes being able to delegate control to someone else who can scaffold, advise, assist, instruct, direct, and help them when needed, as well as being able to take it back whenever they wish – then such methods can be very useful. So can lectures. To all those educational researchers that object to lectures, I ask whether they have ever found them valuable in a conference (and , if not, why did they go to a conference in the first place?). It’s not the pedagogy of lectures that is at fault. It’s the requirement to attend them and the accompanying expectation that people are going to learn what you are teaching as a result. That’s, simply put, empirically wrong. It doesn’t mean that lecturees learn nothing. Far from it. But what you teach and what they learn are different kinds of animal.

Problems with unteaching

It’s really easy to be a bad unteacher – I think that is what Miss Smith is railing against, and it’s a fair criticism. I’m often pretty bad at it myself, though I have had a few successes along the way too. Unteaching and, especially, the pedagogies that result from having done unteaching, are far more likely to go wrong, and they take a lot more emotional, intellectual, and social effort than traditional teaching because they don’t come pre-assembled. They have no convenient structures and processes in place to do the teaching for you.  Traditional teaching ‘works’ even when it doesn’t. If you throw someone into a school system, with all its attendant rewards, punishments, timetables, rules and curricula, and if you give them the odd textbook and assessment along the way, then most students will wind up learning something like what is intended to be taught by the system, no matter how awful the teachers might be. In such a system, students will rarely learn well, rarely persistently, rarely passionately, seldom kindly, and the love of learning will have been squashed out of many of them along the way (survivors often become academics and teachers themselves). But they will mostly pass tests at the end of it. With a bit of luck many might even have gained a bit of useful knowledge or skill, albeit that much will be not just wasted and forgotten as easily as a hotel room number when your stay is over, but actively disliked by the end of it. And, of course, they will have learned dependent ways of learning that will serve them poorly outside institutional systems.

To make things far worse, those very structures that assist the traditional teacher (grades, compulsory attendance, fixed outcomes, concept of failure, etc) are deeply antagonistic to unteaching and are exactly why it is needed in the first place. Unteachers face a huge upstream struggle against an overwhelming tide that threatens to drown passionate learning every inch of the way. The results of unteaching can be hard to defend within a traditional educational system because, by conventional measures, it is often inefficient and time-consuming. But conventional measures only make sense when you are trying to make everyone do the same things, through the same means, with the same ends, measured by and in order to meet the same criteria. That’s precisely the problem.

The final nail in unteaching’s coffin is that it is applied very unevenly across the educational system, so every freedom it brings is counterbalanced by a mass of reiterated antagonistic lessons from other courses and programs. Every time we unteach someone, two others reteach them.  Ideally, we should design educational systems that are friendlier to and more supportive of learner autonomy, and that are (above all else) respectful of learners as human beings. In K-12 teaching there are plenty of models to draw from, including Summerhill, Steiner (AKA Waldorf) schools, Montessori schools, Experiential Learning Schools etc. Few are even close to perfect, but most are at least no worse than their conventional counterparts, and they start with an attitude of respect for the children rather than a desire to make them conform. That alone makes them worthwhile. There are even some regional systems, such as those found in Finland or (recently) British Columbia, that are heading broadly in the right direction. In universities and colleges there are plenty of working models, from Oxford tutorials to Cambridge supervisions, to traditional theses and projects, to independent study courses and programs, to competency-based programs, to PLAR/APEL portfolios, and much more. It is not a new idea at all. There is copious literature and many theoretical models that have stood the test of time, from andragogy to communities of practice, through to teachings from Freire, Illich, Dewey and even (a bit quirkily) Vygotsky. Furthermore, generically and innately, most distance and e-learning unteaches better than its p-learning counterparts because teachers cannot exert the same level of control and students must learn to learn independently. Sadly, much of it is spoiled by coercing students with grades, thereby providing the worst of both worlds: students are forced to behave as the teacher demands in their terminal behaviours but, without physical copresence, are less empowered by guidance and emotional/social support with the process. Much of my own research and teaching is concerned with inverting that dynamic – increasing empowerment and social support through online learning, while decreasing coercion. I’d like to believe that my institution, Athabasca University, is largely dedicated to the same goal, though we do mostly have a way to go before we get it right.

Why it matters

Unteaching is to a large extent concerned with helping learners – including adult learners – to get back to the point at which most children start their school careers – driven by curiosity, personal interest, social value, joy, delight – but that is schooled out of them over years of being taught dependency.  Once misconceptions about what education is for, what teachers do, and how we learn, have been removed, teaching can happen much more effectively: supporting, nurturing, inspiring, challenging, responding, etc, but not controlling, not making students do things they are not ready to do for reasons that mean little to them and have even less to do with what they are learning.

However, though it is an immensely valuable terminal outcome, improved learning is perhaps not the biggest reason for unteaching. The real issue is moral: it’s simply the right thing to do. The greatest value is that students are far more likely to have been treated with the respect, care, and honour that all human beings deserve along the way. Not ‘care’ of the sort you would give to a dog when you train it to be obedient and well behaved. Care of the sort that recognizes and valorizes autonomy and diversity, that respects individuals, that cherishes their creativity and passion, that sees learners as ends in themselves, not products or (perish the thought) customers. That’s a lesson worth teaching, a way of being that is worth modelling. If that demands more effort, if it is more fallible, and if it means that fewer students pass your tests, then I’m OK with that. That’s the price of admission to the unlearning zone.

 

Open Whisper Systems

The Signal protocol is designed for secure, private, encrypted messaging and real-time calling. The protocol, designed by Open Whisper Systems, is used in an increasingly large range of tools (including by Facebook and Google), but their own app is the most interesting application of it. 

The (open, GPL) Signal app is a secure, private messaging and voice chat app for iOS and Android, offering guaranteed and strong end-to-end encryption without having to sign up for a service with dubious privacy standards or further agendas (e.g. Facebook, Apple, Google, Whatsapp, Viber etc). No ads, no account details kept by the company, no means for them (or anyone) to store or intercept messages or calls, the organization is funded by donations and grants. The app uses your phonebook to discover other contacts using Signal – I don’t have many yet, but hopefully a few of my contacts will see this and install it. Call quality seems excellent – as good as Skype used to be before Microsoft maimed it – though I haven’t used it enough yet to assess its reliability. One disadvantage is that, if you have more than one phone and phone number, there seems to be no obvious way to link them together. That’s a particular nuisance on a dual-SIM phone.

It needs a real, verified phone number to get started but, once you have done that, you can link it to other devices too, including PCs (via Chrome or a Chrome-based browser like the excellent Vivaldi), using a simple QR code (no accounts!) so this is a potentially great replacement for things like Whatsapp, Skype, Allo, Viber, etc. No video calling yet, though you can send video messages (and most other things).

 

Address of the bookmark: https://whispersystems.org/#page-top

Get that “new Mac” smell all the time with a $24 scented candle

Some time ago, while comparing the virtues of paper and electronic books, I predicted that the current generation would one day wax lyrical about the smell of a new iPhone much as those from my generation get gooey over the scent of old books.

That day has arrived.

Address of the bookmark: http://www.alphr.com/apple/1004449/get-that-new-mac-smell-all-the-time-with-a-24-scented-candle

Leonard Cohen Dead at 82 – Rolling Stone

Another sad piece of news in a year of much sad news.

To be fair, it does not come as a great surprise. He had been preparing for this for a long time, approaching death with the same style, creativity, and elegance as he approached his life. Cohen said in a recent interview that he was ready to die and, following much the same theme, in his final letter to Marianne Ihlen as she lay dying earlier this year –  ‘We are really so old and our bodies are falling apart and I think I will follow you very soon’. It pleases me greatly that, on his 80th birthday, he followed through on his long-held intention to take up smoking once more.

 

Address of the bookmark: http://www.rollingstone.com/music/news/leonard-cohen-dead-at-82-w449792

Original page

Commons In A Box

Landing-like software from CUNY, based on Buddypress, intended to provide a learning commons with relatively little effort or configuration. It’s a nice bit of packaging, slick, with good collaboration tools and a simple, activity-stream-oriented social network. Commons in a Box is definitely worth looking at if you need a site to support a bottom-up social community or network, and you don’t have a wealth of resources to put into building your own. 

I came across this software because it is being used in the University of Brighton’s newly reborn community site at https://community.brighton.ac.uk which, until it was killed off last year, used to run on Elgg.  I remain a fan of Elgg for building such things, which has a lot more options than BuddyPress available by default, richer access control, and a much more elegant technological design that makes customization more robust and flexible, but this seems to be a great simple solution that just works without demanding much effort, and that, thanks to its WordPress foundations, could be customized to do pretty much anything you’d want a bit of social software to do. 

Address of the bookmark: http://commonsinabox.org/

Understanding the response to financial and non-financial incentives in education: Field experimental evidence using high-stakes assessments

What they did

This is a report by, Simon Burgess, Robert Metcalfe, and Sally Sadoff on a large scale study conducted in the UK on the effects of financial and non-financial incentives on GCSE scores (GCSEs are UK qualifications usually taken around age 16 and usually involving exams), involving over 10,000 students in 63 schools being given cash or ‘non-financial incentives’. ‘Non-financial incentives’ did not stretch as far as a pat on the back or encouragement given by caring teachers – this was about giving tickets for appealing events. The rewards were given not for getting good results but for particular behaviours the researchers felt should be useful proxies for effective study: specifically, attendance, conduct, homework, and classwork. None of the incentives were huge rewards to those already possessing plenty of creature comforts but, for poorer students, they might have seemed substantial. Effectiveness of the intervention was measured in terminal grades. The researchers were very thorough and were very careful to observe limitations and concerns. It is as close to an experimental design as you can get in a messy real-world educational intervention, with numbers that are sufficient and diverse enough to make justifiable empirical claims about the generalizability of the results.

What they found

Rewards had little effect on average marks overall, and it made little difference whether rewards were financial or not. However, in high risk groups (poor, immigrants, etc) there was a substantial improvement in GCSE results for those given rewards, compared with the control groups.

My thoughts

The only thing that does surprise me a little is that so little effect was seen overall, but I hypothesize that the reward/punishment conditions are so extreme already among GCSE students that it made little difference to add any more to the mix.  The only ones that might be affected would be those for whom the extrinsic motivation is not already strong enough. There is also a possibility that the demotivating effects for some were balanced out by the compliance effects for others: averages are incredibly dangerous things, and this study is big on averages.

What makes me sad is that there appears to be no sense of surprise or moral outrage about this basic premise in this report.

dogs being whipped, from Jack London's 'Call of the Wild' It appears reasonable at first glance: who would not want kids to be more successful in their exams? When my own kids had to do this sort of thing I would have been very keen on something that would improve their chances of success, and would be especially keen on something that appears to help to reduce systemic inequalities. But this is not about helping students to learn or improving education: this is completely and utterly about enforcing compliance and improving exam results. The fact that there might be a perceived benefit to the victims is a red herring: it’s like saying that hitting dogs harder is good for the dogs because it makes them behave better than hitting them gently. The point is that we should not be hitting them at all. It’s not just morally wrong, it doesn’t even work very well, and only continues to work at all if you keep hitting them. It teaches students that the end matters more than the process, that learning is inherently undesirable and should only done when there is a promise of a reward or threat of punishment, and that they are not in charge of it.

The inevitable result of increasing rewards (or punishments – they are functionally equivalent) is to further quench any love of learning that might be left at this point in their school careers, to reinforce harmful beliefs about how to learn, and to further put students off the subjects they might have loved under other circumstances for life.  In years to come people will look back on barbaric practices like this much as we now look back at the slave trade or pre-emancipation rights for women.

Studies like this make me feel a bit sick.

 

Address of the bookmark: http://www.efm.bris.ac.uk/economics/working_papers/pdffiles/dp16678.pdf