Speaks for itself.
Address of the bookmark: http://smbc-comics.com/index.php?id=3978
Dumb poll illustrates flaws in objective tests
Given its appearance in Huffpost Weird News, this is a surprisingly acute, perceptive and level-headed analysis of the much-headlined claim that 10% of US college graduates believe Judge Judy serves on the US Supreme Court. As the article rightly shows, this is palpable and scurrilous nonsense. It does show that a few American college graduates don’t know who serves on the Supreme Court (which is not exactly a critical life skill) but, given that over 60% got the answer correct and over 20% picked someone who did formerly serve, the results seem quite encouraging. The article makes the point that Judge Judy is referred to on the poll simply as Judith Sheindlin, that is not the name she is popularly known by, so there is no evidence at all that anyone actually believed her to be a supreme court judge. It was just a wrong and pretty random guess that no one would have got wrong if she had been referred to as ‘Judge Judy’. I’d go further. Most people would only know Judge Judy’s real name if they happened to be fans, in which case they would instantly recognize this as a misdirection and so be able to pick between the three remaining alternatives, one of which even I (with no interest in or knowledge of parochial US trivia) recognize as wrong. So it is quite possible that a large proportion of correct or nearly correct answers were actually due to people watching too much mind-numbing daytime TV. Great.
What it does show in quite sharp relief is how dumb multiple choice questions tend to be. If this were given as a quiz question in a course (not improbable – most are very much like it, and quite a few are worse) it would provide no evidence whatsoever that any given individual actually knew the answer. This is not even a test of recall, let alone higher order knowledge. A wrong answer does not indicate belief that it is true, but a correct answer does not reliably indicate a true belief either. Individually, multiple choice questions are completely useless as indicators of knowledge, in aggregate they are not much better.
As long as they are not used to judge performance or grade students, objective quizzes can be useful formative learning tools. Treated as fun interactive tools, they can encourage reflection, provide a sense of control over the process, and support confidence. They can also, in aggregate, provide oblique clues to teachers about where issues in teaching might lie. In a very small subset of subject matter (e.g. some sub-areas of math problem solving), given enough of them, they might coarsely differentiate between total incompetence and minimal competence. There are also a few ways to improve their reliability – adding a confidence weighting, for example, can help better distinguish between pure guesses and actual semi-recollection, and adaptive quizzes can focus in a bit more on misconceptions, if they are very carefully designed. But, if we are honest, the only reason they are ever used summatively in education or other fields of learning is because they are easy to mark, not because they are reliable indicators of knowledge or performance, and not because they help students to learn: in fact, when given as graded tests, they do exactly the opposite. I guess a secondary driver might be that it is easy to generate meaningful-looking (but largely meaningless) statistics from them. Neither reason seems compelling.
Apart from their uselessness at performing the task they are meant to perform, there are countless other reasons that graded objective tests are a bad idea, from the terrible systemic effects of teaching to the test, to the extrinsic motivation they rely on that kills the love of learning in most learners, to their total lack of authenticity. It is not hard to understand why they are so popular, but it is very hard to understand why teachers and others that see their job as to inspire, motivate and support would do this to students to whom they owe a duty of care.
Address of the bookmark: http://www.huffingtonpost.com/entry/polls-judge-judy-supreme-court_us_569e98b3e4b04c813761bbe8
Space is the Machine
Space is the Machine, a book by Bill Hillier, is available online for free, and is also back in print again after too long an absence. Around 15 or so years ago this book changed how I see the world. As my own well-thumbed paper copy has suffered a lot over the years, and is a very large, heavy object that attracts a lot of dust and not much reading, it is delightful to be able to dip into the pristine electronic version and again be inspired.
This site has each chapter individually downloadable. A full 368-page copy is available at http://discovery.ucl.ac.uk/3881/
The book is as much a work of philosophy as it is of architecture and urban planning (its main subject matter). It incorporates insights from sociology, psychology, anthropology, network theory, linguistics, complexity theory, distributed cognition, systems theory, aesthetics, engineering, ecology, collective intelligence, topology, emergence and more. The ideas it embodies have far broader potential applications than the built environment, including to ways we think about the purpose and practice of education, as well as to more obviously related things like the design of online social applications. In brief, it provides a way of understanding complex human systems and environments as interconnected configurations of structure, objects, time, and movement, in constant dynamic and emergent interplay with abstract, social and psychological phenomena. There are strong echoes of Jane Jacobs (uncited) and Christopher Alexander (cited) in all of this, but it goes farther up and farther in.
I don’t know whether the book and the theories of space syntax it describes impress most architects and urban planners. As I am neither, that’s not the point for me. Whether all the arguments and conclusions make sense in its intended context or not (and some are a bit suspect, even to an outsider like me) this book repeatedly makes strikingly novel connections between diverse and otherwise incommensurate fields, and it constantly provides new perspectives that make the familiar strange and fascinating. It is inspiring stuff.
Address of the bookmark: http://spaceisthemachine.com/
Study shows Facebook spreads nonsense more effectively than fact
An interesting side-effect of the way Facebook relentlessly and amorally drives the growth of its network no matter what the costs: stupidity thrives at the expense of useful knowledge.
This study looks at how information and misinformation spread in a Facebook network, finding that the latter has way more long-term staying power and thus, thanks to EdgeRank and the reification of communication, continues to spread and grow while more ephemeral factual pieces of news disappear from the stream. I suspect this is because actual news has a sell-by date so people move on to the next news. Misinformation of the sort studied (conspiracy theories, etc) has a more timeless and mythic quality that is only loosely connected with facts or events, but it has a high emotional impact and is innately interesting (if true, the world would be a much more surprising place), so it can persist without becoming any more or less relevant. It doesn’t have to spread fast nor even garner much interest at first, because it persists in the network. All it needs to do is wait around for a while – the Matthew Effect and Facebook’s algorithms see to the rest.
There is not much difference between interest in scientific and anti-scientific articles at the start. There is a wave of activity for the first 120 minutes after posting, then a second one 20 hours later (a common pattern). But then the fun starts…
“It’s over the long term that serious differences were observed. While the science news had a relatively short tail, petering out quickly, conspiracy theories tended to grow momentum more slowly, but have a much longer tail. They stick around for a longer period of time, meaning they can reach far more people.
Then there’s another problem with the way Facebook works – the much-discussed echo-chamber effect. This effect is far more active in Facebook than in other networks, with algorithms favouring content from people and groups you regularly interact with. So if you share, Like or even click on conspiracy theories a lot, you’re more likely to be shown them in future, reinforcing the misinformation, rather than challenging it.”
Address of the bookmark: http://www.alphr.com/science/1002377/study-shows-facebook-spreads-nonsense-more-effectively-than-fact
Brain Based Learning and Neuroscience – What the Research Says!
Will Thalheimer provides a refreshing look at the over-hyping of (and quite pernicious lies about) neuroscience and brain-based learning. As he observes, neuroscience is barely out of diapers yet in terms of actual usable results for educators, and those actually researching in the field have no illusions that it is anywhere close yet (though they are very hopeful). What the research says is pretty close to nothing, when it comes to learning practice.
I am a little sceptical about whether neuroscience will ever be really valuable in education. This is not to say it is valueless – far from it. We have already had some useful insights into memory and have a better idea of some of the things that reduce or increase the effectiveness of brain functioning (sleep, exercise, etc), as well as a clearer notion of the mechanisms behind learning. Such things are good to know and can lead to some improvements in learning. The trouble is, though, that most researchers in the area are doing reductive science – seeking repeatable mechanisms and processes that underlie phenomena we see. This is of very little value when dealing with complex adaptive systems and emergence. Stuart Kauffman demonstrates that there are two main reasons reductive explanations fail to give us any help at all with understanding emergent systems: epistemological emergence and ontological emergence. Epistemological emergence means that it is impossible in principle to predict emergent features from constituent parts. Ontological emergence means that completely different kinds of causality occur in and between emergent phenomena than in and between their constituent parts, so knowledge of how the constituent parts work has no bearing at all on higher levels of causality in emergent phenomena. It’s a totally different kind of knowledge.
Knowing how the brain works in education is useful in much the same way that knowing about movements of water molecules in clouds is useful in meteorology. There are insights to be gained, explanations even, but they are of relatively little practical value in predicting the weather, let alone in predicting the precise shape of a specific cloud. Worse, in education, we don’t have a very precise idea of what kind of cloud shape we are seeking, most of the time. In fact, when we act like we do (learning objectives and associated assessment) we usually miss a great deal of the important stuff.
But it is worse than that. Those of us concerned with education are not just predicting or explaining phenomena, but orchestrating them. You can no more extrapolate how to teach from knowing how the brain works than you can extrapolate how to paint a masterpiece from knowing what paint is composed of. They are not even in the same family of phenomena. This doesn’t mean that a painter cannot learn useful things about paint that can assist the process – how fast it dries, its colour fastness, its viscosity, etc, and it does open up potential avenues for designing new kinds of paint. But we still need to know what to do with it once we know that. So, yes, brain science has value in education. Just not that much.
Address of the bookmark: http://www.willatworklearning.com/2016/01/brain-based-learning-and-neuroscience-what-the-research-says.html
Social Influence Bias: A Randomized Experiment
Fascinating article from 2013 on an experiment on a live website in which the experimenters manipulated rating behaviour by giving an early upvote or downvote. An early upvote had a very large influence on future voting, increasing the chances by nearly a third that a randomly chosen piece of content would gain more upvotes in future, with final ratings increased by 25% on average. Interestingly, downvotes did not have the same effect, making very little overall difference. Topics and prior relationships made some difference.
This accords closely with many similar studies and experiments, including a social navigation study I performed about a decade ago, involving clicking on a treasure map, the twist being that participants had to try to guess where, on average, most other people would click. About half the subjects could see where others had already clicked, the about half could not. The participants were aware that the average was taken from those that could not see where others had clicked. The click patterns of each set were radically different…
On closer analysis, of those that could see where others had clicked, around a third of the subjects followed what others had done (as this recent experiment suggests), around a third followed a similar pattern to the ‘blind’ partipants, and around a third actively chose an option because others had not done so – on the face of it this latter behaviour was a bit bizarre, given the conditions of the contest, though it is quite likely that they were assuming just such a bias would occur and acting accordingly.
One thing that might be useful, though very difficult, would be to try to weed out the herd followers and downgrade their ratings. StackExchange tries to do something like this by giving more weight to those that have shown expertise in the past, but it has not fully sorted out the problem of the super-influential that have a lot of good karma as a result of gaming the system, as well as the networks that form within it leading to bias (a problem shared by the less-sophisticated but also quite effective Reddit). At the very least, it might be helpful to introduce a delay to feedback being shown until a certain amount of time has passed or a threshold has been reached.
One thing is certain, though: simple aggregated ratings that are fed back to prospective raters (including those voting in elections) are almost purpose-built to make stupid mobs. As several people have shown, including Surowiecki and Page, crowds are normally only wise when they do not know what the rest of the crowd is thinking.
ABSTRACT
Our society is increasingly relying on the digitized, aggregated opinions of others to make decisions. We therefore designed and analyzed a large-scale randomized experiment on a social news aggregation Web site to investigate whether knowledge of such aggregates distorts decision-making. Prior ratings created significant bias in individual rating behavior, and positive and negative social influences created asymmetric herding effects. Whereas negative social influence inspired users to correct manipulated ratings, positive social influence increased the likelihood of positive ratings by 32% and created accumulating positive herding that increased final ratings by 25% on average. This positive herding was topic-dependent and affected by whether individuals were viewing the opinions of friends or enemies. A mixture of changing opinion and greater turnout under both manipulations together with a natural tendency to up-vote on the site combined to create the herding effects. Such findings will help interpret collective judgment accurately and avoid social influence bias in collective intelligence in the future.
Address of the bookmark: http://www.sciencemag.org/content/341/6146/647.full
Reimagining Online Education ~ Stephen Downes
Stephen Downes provides a typically wise critique of another of those really dumb ‘reimagining education’ pieces that does not reimagine education at all – it just reinforces what is already wrong with it. His points are all sound and worth reflecting on. Though a little strained, I quite like Stephen’s metaphor:
“Education doesn’t more features. It needs authentic propulsion and sound aerodynamic design. Sadly, most educational professionals don’t study aerodynamics, they study ornithology.”
I could extend the metaphor a little further. While many educators are stuck on ornithology (and some stopped looking any further than the archeopteryx), I think many educational researchers, at least in e-learning, are looking at more ways to tweak propellor-driven biplanes or trying to make airport check-ins more efficient. Some are looking at jet planes and rocket ships. Some are exploring helicopters and hovercraft. Perhaps a few are wondering how to build personal teleporters.
What education actually needs, though, is a thorough critical reconsideration of the entire transport system, taking into consideration what people want from it, why they choose to travel in the first place, their levels of comfort, their levels of risk, what the constraints are, what effects it has on the broader ecosystem, how it affects people’s psyches, how it stimulates them, and how it changes social patterns, amongst many other things. There’s a really important place for bicycles, buses, trains, ships, boats, footpaths, skateboards, snowmobiles, gliders, skis, horse-drawn buggies, hoverboards and all the rich diversity of transportation devices and infrastructure we have invented and will invent. It’s not one science: it’s a host of technologies and, above all, it’s a system invented by and for humans.
Bearing that in mind, education really needs a better metaphor than travel from A to B. At the very least, there is an indefinitely large range of more important and interesting stuff happening between A and B than ever happens at the destination, there’s a great deal of important stuff to say about the comfort and stimulation of the passengers in transit, and, often, ‘B’ is not where they want or need to be anyway.
Address of the bookmark: http://www.downes.ca/post/64800
Juniper Networks backdoor confirmed, password revealed, NSA suspected
Yet another reason to be deeply concerned for privacy. The NSA or some other agency has embedded a backdoor into the firewalls that ‘protect’ a great many organizations, allowing them (and now the whole world) to decrypt supposedly private communications, virtually undetectably and at will.
This kind of vulnerability might affect any closed-source product, but it is particularly worrying when it exists at such a crucial node in the network infrastructure. AU’s own VPN has been moving across to Juniper’s Junos Pulse over the past month or two, and AU has been increasingly shifting to closed-source, proprietary products from US companies (and, in the case of email and webinars, using services that are actually based in the US). This is a truly terrible idea. Open source products are not invulnerable to such manipulation, but the chances of finding flaws are at least thousands of times greater than in closed-source products like this, and it is possible for individuals to fix them, no matter how old the product. Given other open-source advantages like vendor-independence, control, capacity to be altruistic, and innate flexibility, it is hard to understand why anyone would entrust their network infrastructure to a proprietary, closed-source company.
Address of the bookmark: http://boingboing.net/2015/12/21/juniper-networks-backdoor-conf.html
Your iPhone Is Ruining Your Posture — and Your Mood – The New York Times
Are you sitting up straight as you are reading this? Good for you!
This is a report on something that has been bothering me for some time. After decades of taking inordinate care over the ergonomics of computers and making sure that I did all the right things to avoid the worst effects, tablets and smartphones have snuck in to my life and wrecked all that hard work. The article focuses mainly on the posture we adopt when using smart devices. This little snippet is worth thinking on:
“The average head weighs about 10 to 12 pounds. When we bend our necks forward 60 degrees, as we do to use our phones, the effective stress on our neck increases to 60 pounds — the weight of about five gallons of paint. “
As well as the obvious physical issues, the report describes some of the psychological ones, and they are doozies. The hunched posture makes us more depressed, less assertive, with less self-esteem, and with greater anxiety. It can, apparently, even make us more forgetful. This is happening in small ways but at a huge scale: it must be having a significant effect on societies all the way across the planet. Scary.
I doubt that many of us are willing to give up our devices, so we need to work on ways of reducing the ill effects. Smaller devices cause more hunching than larger ones. However I have found larger ones play more havoc with my hand muscles (even the super-light iPad Air 2), so it’s a case of swings and roundabouts. And it doesn’t help much to use stands, because then all the old problems with fixed-screen computers come back to haunt you, this time on even small and brighter screens. I did find a hand-strap worked fairly well for me for a while, until it broke (not a good thing with an expensive device).
The main thing, I believe, is to vary your posture, grip and reading angle/distance as much as you can, as often as you can. If you have been hunched, it also helps to stretch and contract your shoulder blades, and roll your head a bit from time to time. As a musician, I find playing an instrument is not a bad way of putting your fingers and wrists in different positions for a while, but it can be over-done. I’m also a fan of eye exercises: alternately look at very near and very far things for a while every now and then.
We should be teaching this stuff in schools and in public safety videos and posters. It seems to me that this is a massive and increasingly ubiquitous health problem that is all the worse for usually being quite subtle. We are changing not only our physical form but our collective psyche. Throw the known harmful effects of Facebook and its kin into the mix and it’s a recipe for a slow and sad disaster. And we’ll be too depressed to care that it has happened.
Address of the bookmark: http://www.nytimes.com/2015/12/13/opinion/sunday/your-iphone-is-ruining-your-posture-and-your-mood.html?_r=0
Virtual Canuck | Teaching and Learning in a Net-Centric World
Terry Anderson has, after many years, moved his much-loved Virtual Canuck site to a shiny new system with its own domain, and it’s looking very good.
There’s masses of stuff here for anyone with an interest in distance and online education, and quite a few other things that relate to Terry’s diverse interests, from music to Unitarianism. Don’t miss his latest post on the new IRRODL special issue on MOOCs – some great commentary on and summaries of articles.
Address of the bookmark: http://virtualcanuck.ca/