Evaluating assessment

Posted , updated on

Exam A group of us at AU have begun discussions about how we might transform our assessment practices, in the light of the far-reaching AU Imagine plan and principles. This is a rare and exciting opportunity to bring about radical and positive change in how learning happens at the institution. Hard technologies influence soft more than vice versa, and assessments (particularly when tied to credentials) tend to be among the hardest of all technologies in any pedagogical intervention. They are therefore a powerful lever for change. Equally, and for the same reasons, they are too often the large, slow, structural elements that infest systems to stunt progress and innovation.

Almost all learning must involve assessment, whether it be of one’s own learning, or provided by other people or machines. Even babies constantly assess their own learning. Reflection is assessment. It is completely natural and it only gets weird when we treat it as a summative judgment, especially when we add grades or credentials to the process, thus normally changing the purpose of learning from achieving competence to achieving a reward. At best it distorts learning, making it seem like a chore rather than a delight, at worst it destroys it, even (and perhaps especially) when learners successfully comply with the demands of assessors and get a good grade. Unfortunately, that’s how most educational systems are structured, so the big challenge to all teachers must be to eliminate or at least to massively reduce this deeply pernicious effect. A large number of the pedagogies that we most value are designed to solve problems that are directly caused by credentials. These pedagogies include assessment practices themselves.

With that in mind, before the group’s first meeting I compiled a list of some of the main principles that I adhere to when designing assessments, most of which are designed to reduce or eliminate the structural failings of educational systems. The meeting caused me to reflect a bit more. This is the result:

Principles applying to all assessments

  • The primary purpose of assessment is to help the learner to improve their learning. All assessment should be formative.
  • Assessment without feedback (teacher, peer, machine, self) is judgement, not assessment, pointless.
  • Ideally, feedback should be direct and immediate or, at least, as prompt as possible.
  • Feedback should only ever relate to what has been done, never the doer.
  • No criticism should ever be made without also at least outlining steps that might be taken to improve on it.
  • Grades (with some very rare minor exceptions where the grade is intrinsic to the activity, such as some gaming scenarios or, arguably, objective single-answer quizzes with T/F answers) are not feedback.
  • Assessment should never ever be used to reward or punish particular prior learning behaviours (e.g. use of exams to encourage revision, grades as goals, marks for participation, etc) .
  • Students should be able to choose how, when and on what they are assessed.
  • Where possible, students should participate in the assessment of themselves and others.
  • Assessment should help the teacher to understand the needs, interests, skills, and gaps in knowledge of their students, and should be used to help to improve teaching.
  • Assessment is a way to show learners that we care about their learning.

Specific principles for summative assessments

A secondary (and always secondary) purpose of assessment is to provide evidence for credentials. This is normally described as summative assessment, implying that it assesses a state of accomplishment when learning has ended. That is a completely ridiculous idea. Learning doesn’t end. Human learning is not in any meaningful way like programming a computer or storing stuff in a database. Knowledge and skills are active, ever-transforming, forever actively renewed, reframed, modified, and extended. They are things we do, not things we have.

With that in mind, here are my principles for assessment for credentials (none of which supersede or override any of the above core principles for assessment, which always apply):

  • There should be no assessment task that is not in itself a positive learning activity. Anything else is at best inefficient, at worst punitive/extrinsically rewarding.
  • Assessment for credentials must be fairly applied to all students.
  • Credentials should never be based on comparisons between students (norm-referenced assessment is always, unequivocally, and unredeemably wrong).
  • The criteria for achieving a credential should be clear to the learner and other interested parties (such as employers or other institutions), ideally before it happens, though this should not forestall the achievement and consideration of other valuable outcomes.
  • There is no such thing as failure, only unfinished learning. Credentials should only celebrate success, not punish current inability to succeed.
  • Students should be able to choose when they are ready to be assessed, and should be able to keep trying until they succeed.
  • Credentials should be based on evidence of competence and nothing else.
  • It should be impossible to compromise an assessment by revealing either the assessment or solutions to it.
  • There should be at least two ways to demonstrate competence, ideally more. Students should only have to prove it once (though may do so in many ways and many times, if they wish).
  • More than one person should be involved in judging competence (at least as an option, and/or on a regularly taken sample).
  • Students should have at least some say in how, when, and where they are assessed.
  • Where possible (accepting potential issues with professional accreditation, credit transfer, etc) they should have some say over the competencies that are assessed, in weighting and/or outcome.
  • Grades and marks should be avoided except where mandated elsewhere. Even then, all passes should be treated as an ‘A’ because students should be able to keep trying until they excel.
  • Great success may sometimes be worthy of an award – e.g. a distinction – but such an award should never be treated as a reward.
  • Assessment for credentials should demonstrate the ability to apply learning in an authentic context. There may be many such contexts.
  • Ideally, assessment for credentials should be decoupled from the main teaching process, because of risks of bias, the potential issues of teaching to the test (regardless of individual needs, interests and capabilities) and the dangers to motivation of the assessment crowding out the learning. However, these risks are much lower if all the above principles are taken on board.

I have most likely missed a few important issues, and there is a bit of redundancy in all this, but this is a work in progress. I think it covers the main points.

Further random reflections

There are some overriding principles and implied specifics in all of this. For instance, respect for diversity, accessibility, respect for individuals, and recognition of student control all fall out of or underpin these principles. It implies that we should recognize success, even when it is not the success we expected, so outcome harvesting makes far more sense than measurement of planned outcomes. It implies that failure should only ever be seen as unfinished learning, not as a summative judgment of terminal competence, so appreciative inquiry is far better than negative critique. It implies flexibility in all aspects of the activity. It implies, above and beyond any other purpose, that the focus should always be on learning. If assessment for credentials adversely affects learning then it should be changed at once.

In terms of implementation, while objective quizzes and their cousins can play a useful formative role in helping students to self-assess and to build confidence, machines (whether implemented by computers or rule-following humans) should normally be kept out of credentialling. There’s a place for AI but only when it augments and informs human intelligence, never when it behaves autonomously. Written exams and their ilk should be avoided, unless they conform to or do not conflict with all the above principles: I have found very few examples like this in the real world, though some practical demonstrations of competence in an authentic setting (e.g. lab work and reporting) and some reflective exercises on prior work can be effective.

A portfolio of evidence, including a reflective commentary, is usually going to be the backbone of any fair, humane, effective assessment: something that lets students highlight successes (whether planned or not), that helps them to consolidate what they have learned, and that is flexible enough to demonstrate competence shown in any number of ways. Outputs or observations of authentic activities are going to be important contributors to that. My personal preference in summative assessments is to only use the intended (including student-generated) and/or harvested outcomes for judging success, not for mandated assignments. This gives flexibility, it works for every subject, and it provides unquivocal and precise evidence of success. It’s also often good to talk with students, perhaps formally (e.g. a presentation or oral exam), in order to tease out what they really know and to give instant feedback. It is worth noting that, unlike written exams and their ilk, such methods are actually fun for all concerned, albeit that the pleasure comes from solving problems and overcoming challenges, so it is seldom easy.

Interestingly, there are occasions in traditional academia where these principles are, for the most part, already widely applied. A typical doctoral thesis/dissertation, for example, is often quite close to it (especially in more modern professional forms that put more emphasis on recording the process), as are some student projects. We know that such things are a really good idea, and lead to far richer, more persistent, more fulfilling learning for everyone. We do not do them ubiquitously for reasons of cost and time. It does take a long time to assess something like this well, and it can take more time during the rest of the teaching process thanks to the personalization (real personalization, not the teacher-imposed form popularized by learning analytics aficionados) and extra care that it implies. It is an efficient use of our time, though, because of its active contribution to learning, unlike a great many traditional assessment methods like teacher-set assignments (minimal contribution) and exams (negative contribution).  A lot of the reason for our reticence, though, is the typical university’s schedule and class timetabling, which makes everything pile on at once in an intolerable avalanche of submissions. If we really take autonomy and flexibility on board, it doesn’t have to be that way. If students submit work when it is ready to be submitted, if they are not all working in lock-step, and if it is a work of love rather than compliance, then assessment is often a positively pleasurable task and is naturally staggered. Yes, it probably costs a bit more time in the end (though there are plenty of ways to mitigate that, from peer groups to pedagogical design) but every part of it is dedicated to learning, and the results are much better for everyone.

Some useful further reading

This is a fairly random selection of sources that relate to the principles above in one way or another. I have definitely missed a lot. Sorry for any missing URLs or paywalled articles: you may be able to find downloadable online versions somewhere.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. Retrieved from https://www.jhsph.edu/departments/population-family-and-reproductive-health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-learning.pdf

Boud, D. (2007). Reframing assessment as if learning were important. Retrieved from https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in organizational change and development, 1, 129-169.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3/4), 325-346.

Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3(3), 220-233.

Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes (Kindle ed.). Mariner Books. (this one is worth forking out money for).

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. Retrieved from http://www.alfiekohn.org/blogs/personalized/ (check out Alfie Kohn’s whole site for plentiful other papers and articles – consistently excellent).

Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-Determination research (pp. 183-203). Rochester, NY: The University of Rochester Press.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications. (may be worth paying for if such things interest you).

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Cairo: Ford Foundation. http://www.managingforimpact.org/sites/default/files/resource/outome_harvesting_brief_final_2012-05-2-1.pdf.

The perfect Ploughman’s Lunch

Posted , updated on
ploughman's lunch with bread, sausage, stilton, salad, pickled onion, etc, served with a pint of bitter on a white place
A very acceptable ploughman’s lunch,
served in the ancient town of Lewes.

Life in Canada is, generally speaking, wonderful when compared with life in the UK but, as well as the loved ones I miss, there are a few everyday things that I yearn for from the old country, most often connected with food or drink. I am reminded on this sunny day in late August of the meal that I miss most: the Ploughman’s Lunch. I almost wrote ‘the traditional Ploughman’s Lunch’ but Ploughman’s Lunches are not really traditional, though they are loosely based on what rural workers ate and drank for centuries. The term and the dish were invented by a cheese marketing board in the late 1950s, and enthusiastically promoted for sale in pubs – usually with a glass of ale or cider – as a more substantial and satisfying alternative to traditional pies and sandwiches, but just as easy for inexpert pub staff with no proper kitchen to prepare. They have been almost ubiquitous in English pubs for my entire life. I know of a handful of pubs in Greater Metro Vancouver that purport to serve them but, without exception, even when they do other British dishes like bangers and mash, toad in the hole, Shepherd’s Pie, and so on well, they are not even close to the real thing and are almost guaranteed to disappoint. A couple of ‘British’ pubs in Victoria get pretty close, with something you might find in a mediocre chain pub in the UK, but none do it right.

So here, in case any pub owners in Vancouver or the surrounding area ever get to read it, is how you do it right…

Two or three really thickly cut slices or wedges of crusty bread (but not the sort of crust that will break your teeth). Not toast (though very light toasting may be OK if the bread is not completely fresh), not thin-sliced bread, no fancy flavours or additions, definitely not wraps or flatbreads. The perfect bread is an English cottage loaf or similar, wholemeal or otherwise. Sourdough is acceptable, a baguette works if it is really fresh, or a country loaf inspired by French, Italian, or similar traditions can substitute well. A nice roll – ciabatta or similar – may do at a pinch (that is what is in the image above, and it was quite pleasant). Not optional.

A small tub of soft (but not too soft) cultured butter (never margarine, never uncultured butter – almost all butter served in the UK is cultured, but it is not the default here in Canada). A handful of foil packets of Anchor butter or similar are acceptable and commonplace (see image above). Not optional.

A substantial wedge of sharp, well-aged Cheddar or similar hard cheese. Shouldn’t be the shrink-wrapped supermarket variety unless you cannot find anything better. Stilton is a good alternative/addition (as in the image above). Not optional.

A wedge or two of Melton Mowbray pork pie (the best) or other meat product such as the sausage in the image above. Optional.

A generous, thick-cut slice or two of slightly chewy baked ham. Never, ever, ever, substitute the pasty gelatinous mechanically recovered slices that come in plastic boxes at your local supermarket, or turkey slices (though a good chunk of smoked turkey from a delicatessen works well), or thin-sliced charcuterie meats, pastrami, salami, etc. Optional.

A dob of intensely hot Coleman’s English mustard or similar. Only needed (in tiny quantities) if you are having the ham, pork pie, sausage, etc. Don’t substitute Dijon, German, grainy, or other mild, vinegary alternatives unless you really can’t stand the intensity of proper English mustard.

A pickled onion or two. Not optional. I prefer strong pickled onions but medium strength will do. Do not substitute cocktail onions, or mild pickled onions, and especially do not even consider substituting dill pickles or gherkins. How could you even think of such a thing? I’m looking at you, Vancouver pubs.

Plenty of proper, dark brown, crunchy, chunky, richly flavoured Branston pickle (not the weird light-brown goop that tends to be sold in many Canadian stores – many supermarkets now stock the real thing, made by Crosse & Blackwell, usually in the British part of the ‘ethnic food’ section). Heinz, Waitrose, or Marks & Spencer Ploughman’s pickle will do at a pinch. Not optional. Do not substitute chutney or other sweet goo, especially if flavoured with cinnamon or other strong, fragrant spices. A really good, sharp, crunchy mango chutney, though, with a not-too-sweet sauce might be OK if you really hate Branston.

Some good, sharp, bright, chunky, crunchy piccalilli. Optional (good if you are having the ham or pork pie).

A fairly plain, leafy salad with lettuce, tomatoes, some red or other mild onion, maybe some cucumber, perhaps a sprig or two of parsley as a garnish. Go light on the dressing, if you use any at all. Mayonnaise may be provided on the side. Not optional. Do not substitute coleslaw, exotic leaves, potato salad, caesar salad, etc. Keep it simple, keep the ingredients distinct.

A few other ingredients may be added or, sometimes, substituted to taste, such as a Scotch egg, a gherkin or two, perhaps some coleslaw, maybe a British banger or similar sausage, maybe a boiled or pickled egg, possibly some black pudding or liver sausage/pate, perhaps a bit of game pie instead of pork pie; certainly an alternative or additional hard or semi-hard cheese or two (Caerphilly, Red Leicester, Wensleydale, Cheshire, Gloucester, etc); perhaps a slice or two of thickly cut cold roast beef (horseradish optional, otherwise mustard) or cold roast lamb (with mustard or mint sauce), or maybe a chicken or turkey drumstick (lovely with Branston) instead of ham; perhaps a chutney (as well as, not instead of, Branston), maybe some pickled beetroot, perhaps some watercress, radish, cress, celery, etc in the salad.

Never serve any ingredients hot, though the bread can benefit from being a little warm.

Don’t overdo it. The best Ploughmans tend to keep things fairly simple, with two or three main proteins, in chunks or wedges or thick slices, good bread, a simple salad, pickled onion, and Branston or Ploughman’s Pickle, with only a smattering of signature embellishments to complement the main centrepieces.

Absolutely essential, and not optional, it must be paired with the right drink…

The perfect accompaniment is real English bitter, pulled from a barrel (cask), never from a keg, bottle, or can, served at cellar temperature (not warm, not room temperature, certainly not cold, just a little cool), with the lightest ring of froth, not completely flat but with no visible bubbles (the texture of velvet), and no more than 4% alcohol. A mild or IPA will do just as well, though remember that, in England, a really strong IPA is around 4.5%. You could substitute Guinness or similar stout if you wish. If that’s not possible, a cask-style nitro can (Kilkenny is the most common brand sold here) is better than nothing, though not ideal. Avoid anything with bubbles. If you don’t like beer, scrumpy (the real stuff, never fizzy, always cloudy, always dry), or a proper French cider or similar will do. Beware the alcohol content of real scrumpy, if you can find it here: you can drink it as easily as orange juice, and it hardly tastes alcoholic at all, but it will flatten you faster than hard liquor. Red wine is acceptable. If you don’t want alcohol, a glass of real lemonade is not a bad substitute, or perhaps a jug of water with a slice of cucumber or mint, or maybe a lime juice cordial or lemon barley water. Avoid anything sweet or fizzy or very strongly flavoured, unless you are sure the flavour will complement the dish (red wine is normally good because it cuts through the fatty, protein rich ingredients, much like the pickle components of the meal).

Serve it on a large, plain, white china plate, with a knife, fork, and cloth napkin. Use a wooden platter if you must, but only if you are a stockbroker, social media influencer or advertising executive. Do not use slate, stone, or fancy porcelain.

Assemble and eat the ingredients in any order or combination you like. Experiment with different combinations. Use your fingers for most of it (including pickled onions, chunks of cheese, pie, meat, sausage, etc as well as the bread). Expect things to get messy. You’ll probably need that cloth napkin.

Eat it in a leafy grassy pub garden on a lazy sunny, but not too hot, day, if possible surrounded by trees, hedges, or a crumbling brick wall. A babbling brook helps. If possible, sit at an untreated wooden bench. Beware of sparrows. In times of covid, eating inside is inadvisable anyway but, if you must, find a sheltered nook.

Do not, under any circumstances, add TV screens, piped music, or music to dance to. A little live light jazz, folk, or classical music is acceptable if you can still hear the bees buzzing in the garden. If you can add the very lightest hint of cigarette and/or cigar smoke in the air, that’s a plus.

Do not expect to tip your server, do not expect your server to ask if you are still working on it, do not expect your server to clear away your plate while you are still chewing. In fact, do not necessarily expect a server at all: you might have to order and pick it up from the bar, along with your beer. This is fine.

Add perfect company, relax, and enjoy. If you finish it before you’ve had time to order a second pint then you are eating too fast, drinking too slow, or there’s something wrong with the portion size. Take your time. This is a meal to be savoured, not devoured.

Kafkaesque and Orwellian technology design

Posted

Death certificate of undead RomanianI am much indebted to the Romanian legal system for the examples it repeatedly provides of hard (rigid, inflexible, invariant) technologies enacted by human beings without the inconvenience, lack of accountability, or cost of actual machinery. I have previously used examples from two cases in which Romanian mayoral candidates were elected to office despite being dead (here, and – though the link seems dead and unarchived so I cannot confirm it – here). This, though, is the best example yet. Despite the moderately compelling evidence he provided to the court that he is alive (he appeared in person to make his case) the court decided that Constantin Reliu, 63, is in fact, still dead, upholding its earlier decision on the subject. This Kafkaesque decision has had some quite unpleasant consequences for Reliu, who cannot get official documents, access to benefits, and so on as a result. Romania is, of course, home to Transylvania and legends of the undead. Reliu is maybe more unalive than undead, though I guess you could look at it either way.

The misapplication of hard technology

The mechanical application of rules, laws, and regulations is rarely a great idea. One advantage of human-enacted hard technologies over those that are black-boxed inside machines, though, is that, on the whole and notwithstanding the Romanian legal system, the workings of the machine are scrutable and can more easily be adapted. Even when deliberations occur (intentionally or not) in camera, the mechanism is clear to participants, albeit that it is rare for all participants to be equally adept in implementing it.

Things are far worse when such decisions are embedded in machines, as a great many A-level students in the UK are discovering at the moment. Though the results are appalling and painful in every sense – the algorithm explicitly reinforces existing inequalities and prejudices, notably disadvantaging racial minorities and poorer students – it is hard not to be at least a little amused by, say, the fact that an 18-year-old winner of the Orwell Prize for her dystopian story about the use of algorithms to sort students according to socio-economic class had her own A-level mark (in English) reduced by exactly such an algorithm for exactly such a reason. Mostly, though, such things are simply appalling and painful, with little redeeming irony apart from the occasional ‘I never thought leopards would eat MY face‘ moment. Facebook, to pick an easy target, has been unusually single-minded in its devotion to algorithms that divide, misinform, demean, and hurt, since its very beginnings. The latest – misinforming readers about Covid-19 – has had direct deadly consequences though, arguably, its role in electing the antichrist to the US presidency was way more harmful.

The ease with which algorithms can and, often, must be embedded in code is deeply beguiling. I know because I used to make extensive use of them myself, with the deliberate intent of affecting the behaviour of people who used my software. My intentions were pure: I wanted to help people to learn, and had no further hidden agendas. And I was aware of at least some of the dangers. As much as possible, I tried to move the processing from the machine to the minds of those using it and, where I could not do that, I tried to make the operation of my software as visible, scrutable, and customizable as possible (why do we so often use the word ‘transparent’ when we mean something is visible, by the way?). This also made them far more difficult to use – softness in technologies always demands more active thought and hard work in users. None-the-less, my apps were made to affect people because – well – why else would there be any point in doing it?

Finding the right balance

The Landing (my most recent major software project) is, on the face of it, a bit of an exception. It is arguably fortunate that some of my early plans for it, involving algorithmic methods like collaborative filtering and social navigation, failed to come to fruition, especially as one of the main design principles on which the Landing was based was to make the site as neutral and malleable as possible. It was supposed to be by and for its users, not for any other purpose or person, not even (like an LMS) to embed the power structures of the university (though these can emerge through path dependencies in groups). However, it is impossible to avoid this kind of shaping altogether. The Landing has quite a few structural elements that are determined by algorithms – tag clouds, recommended content, social network mining for ‘following’ recommendations, etc – but it also embodies values in its very design. Its menu system, for instance, is based on work Terry Anderson and I did that split the social world into networks, groups, and sets, and is meant to affect how people engage. It has a whole bunch of defaults, from default permissions to default notification settings, that are consciously intended to shape behaviour. When it does not do that kind of shaping, though, things can be much worse. The highly tool-centric and content-neutral design that puts the onus on the individual person to make sense of it is one of the reasons it is a chaotic jumble that is difficult to use, for instance

We need some hardness in our technologies – constraint is vital to creation, and many things are better done by machines – but each individual’s needs for technology hardening are different from those of pretty much every other. Hardness in machines lets us do things that are otherwise impossible, makes many things easier, quicker, more reliable, and more consistent. This can be a very good thing but it is just as easy – and almost inevitable – to harden some things that would be better done by people, or that actively cause harm, or that should be adapted to individual needs. We are all different, one size does not fit all.

Openness and control

It seems to me that a fundamental starting point for dealing with the wrong kind of hardness is knowing what and how things are being hardened, and to be capable of softening them if necessary. This implies that:

  • openness is essential: we must be able to see what these things are doing;
  • the ability to make changes is essential: we must be able to override or modify what they do.

Actually messing with algorithms is complex, and it’s usually complicated, which is an unholy mix. It can also be dangerous, at best breaking the machine and at worst making it more harmful than ever. The fact that we can scrutinize and make changes to our tools does not mean that we should, not that we are actually able to exert any meaningful amount of control, unless we have the skills, time, energy, and mandate to do so. Moreover, there are often reasons we should not do so: for instance, a lot of crowd-based systems would not work at all if individual users could adjust how they work, modified software can be used to cause deliberate harm, and so on. It seems to me, though, that having such issues is far preferable to not knowing how we are affected, and not being able to fix it. Our technologies must be open, and they must be controllable, if we are not to be lost in the mire of counter-technologies, Monkeys’ Paws, and malicious machines that increasingly define our lives today.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6368257/kafkaesque-and-orwellian-technology-design

I am disgusted, outraged, furious, sickened, and irritated by this perverse research (actually, no I am not at all)

Posted

I am disgusted, outraged, furious, sickened, and irritated by this perverse research (actually, no I am not at all)

https://www.pnas.org/content/114/28/7313

   Image of disgust  I am really not at all offended in any way by this well-conducted, clearly reported, very interesting research into what the authors describe as ‘moral contagion’. The actual article title is ‘Emotion shapes the diffusion of moralized content in social networks‘, by Brady et al, from 2017. If the research is valid (it seems solid), I should probably get quite a few more retweets of this bookmark than usual when it gets posted to Twitter. The findings are fascinating, and help to partly explain the success of awful, awful, awful people and their ideas in social media such as Twitter, Faecesbook, and the like.

Abstract

Political debate concerning moralized issues is increasingly common in online social networks. However, moral psychology has yet to incorporate the study of social networks to investigate processes by which some moral ideas spread more rapidly or broadly than others. Here, we show that the expression of moral emotion is key for the spread of moral and political ideas in online social networks, a process we call “moral contagion.” Using a large sample of social media communications about three polarizing moral/political issues (n = 563,312), we observed that the presence of moral-emotional words in messages increased their diffusion by a factor of 20% for each additional word. Furthermore, we found that moral contagion was bounded by group membership; moral-emotional language increased diffusion more strongly within liberal and conservative networks, and less between them. Our results highlight the importance of emotion in the social transmission of moral ideas and also demonstrate the utility of social network methods for studying morality. These findings offer insights into how people are exposed to moral and political ideas through social networks, thus expanding models of social influence and group polarization as people become increasingly immersed in social media networks.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6362693/i-am-disgusted-outraged-furious-sickened-and-irritated-by-this-perverse-research-actually-no-i-am-not-at-all

I am a dunce

Posted , updated on
Cat wearing dunce cap

A couple of nights ago I was trying to set up a new WordPress site for a family member and failing miserably. After a while I figured that I had a problem with a rather subtle kind of database corruption, caused by me taking a few shortcuts when transferring some databases from an old server to this one without updating the tables to match the slightly newer version database engine on the new system (my first mistake of many). No problem, I thought: this is not my first database rodeo. Being the cautious kind, despite having two quite independent backup systems seemingly running, I used rsync to make a copy of all the (at this point working) databases before proceeding. You can never be too careful.

Inevitably, things didn’t go according to plan.

While trying to repair it, I managed to destroy almost every database on the system, including the all-important MySQL database itself. Turns out MySQL’ s repair tools are a bit vicious and/or I don’t know enough about how to use them. Anyway, no problem, I could just restore my fresh backup and try again.

Nope.

In fact, MySQL itself would not even start. I can only guess that tables that are in current use are skipped by rsync, because a few were OK but most were missing. This is something I need to investigate further because it has worked for me before: maybe I missed a flag or something.

No problem. I had two alternative restore options. I’m a belt, braces, and elasticated waist kind of a guy.

My first-line backup turned out never to have been saved all the database tables, though it had saved some of them, and almost everything else. That took a while to figure out. I guess the reason might be similar to the issue with rsync but it is another thing I need to investigate.

No worries, I had my second line backup ready, created by a smart WordPress plug-in that exports everything on a nightly basis in friendly zipped up SQL format to a totally different server.

At least, it did do that until late 2018.

And it didn’t include the MySQL system tables.

Not wanting to lose well over a year and a half’s worth of changes, I went through an increasingly desperate range of potential fixes, using the in-built tools, playing with removal of various system tables (some of which can be recreated when needed, some not, as it turns out), updating MySQL, and culminating in reinitializing the MySQL database itself. That got the database management system up and running again so I set about recreating the users and permissions – luckily not a huge issue as I had sensibly recorded the relevant info in a separate file. I then tried many ways to combine the backups I had to fix the errant databases, but all failed.

But at least I had my 2018 backup. So I restored that to the site – which worked fine, and I even remembered to allow for database version changes – and then resyndicated the last 20 months or so of posts from the Landing, which is normally where I post things first.

Oops.

I am sorry, Twitter followers, for the dozens of notifications you then received in the middle of the night (did I mention that it was considerably after midnight by this time?) because I had forgotten to switch off the WordPress tool that automatically tweets new posts. I don’t know whether it helped that I then deleted all of those tweets from my Twitter account. Probably not too much: the notifications had already been sent.

I eventually found a previously forgotten proper SQL backup I’d made back in January this year, as a precaution some weeks before moving the sites to their new home. I was then able to resyndicate at least most of my posts that had originated on the Landing since then, remembering to switch off the automated tweeting tool this time.

I have lost a fair number of tweaks and pages that I had created or modified on the site itself over the past few months, and I expect I’ll be finding quite a few of these over the next little while, but they are mostly not too troublesome to recreate. I’m going to need to work out how to deal with media files (that were unaffected but that have become divorced from WordPress) but I think that shouldn’t be too painful. A few comments from visitors have been permanently lost, but otherwise all seems (touch wood) to be more or less OK now. Fortunately, the owners of the other sites I host on my server haven’t been using them much since January, so (though a few comments may have been lost), they are not too cross.

Have I learned anything useful from this?

Probably not. Not enough, anyway, and likely not persistently. Technical skills are very transient and fade fast when you don’t use them. I had to look up many things that used to be second nature to me this time round. Even if I had remembered well, the technologies I was using had evolved. When you develop the skills to be part of a machine they are of limited use when the machine itself changes. This, as it happens, had a lot to do with the problem I was trying to fix in the first place. As in all things technological, including all things educational, the tools matter far less than the assembly and orchestration of them.

I have managed database management systems for approaching 30 years. In fact, if you count DBaseII/III/IV systems on personal computers, it’s closer to 35 years. I have suffered as a result of carelessness, stupidity, bugs, malice, bad software companies and natural decay enough times to know that you should regularly restore your backups and go through the rest of your recovery procedures on a regular basis, and certainly after every significant change (a version upgrade, for instance). In fact, I teach database management at graduate level and I enthusiastically correct students who forget it. But managing actual databases is very much an occasional hobby for me now, and I don’t have the time, money, or patience to do what I know should be done. A full restore is a big job that carries its own risks, so I try to avoid it, and take risks rather than lose a good night or two of work.

I’m contemplating containerization so that I only have to deal with big self-contained chunks though (with my current setup and limited experience of containers) that is a daunting option. I’ve considered paying someone else but, as a lot of the reason for doing this is not just the ability to control my own systems but to learn how to do so (I hate teaching things when my own skills are rusty) that’s a bit of a cop-out. I am vaguely wondering about setting up a cluster, though that’s a great deal more effort and only protects against a limited, if important, range of problems (it would have helped here, because I would have switched off mirroring before doing this, and could have reversed it when things went wrong). I welcome any thoughts anyone might have on the subject! Any rock solid (Linux) backup tools that are more straightforward than Bacula? Any smart strategies for keeping systems safe without major effort or skill needed?

Letting go and staying close: presentation to GMR Institute of Technology, India, August 2020

Posted

Letting go and staying close

Here are my slides from a presentation I gave to GMR Institute of Technology, Rajam of Srikakulum District, Andhra Pradesh State, India, last week. I gave the presentation from a car, parked in a camp site in the midst of British Columbia, surrounded by mountains and lakes and forest, taking advantage of a surprisingly decent 4G connection via an iPad. It was sadly not interactive, but I hope that those present learned something useful (even if, as my presentation emphasized, they did not learn what I intended to teach).

The general gist of it is that, when teaching online, we need to let go because the in-person power we have in a classroom simply isn’t there. There are other consequences – the need to build community, to demonstrate caring, to accept and value the context of the learner, to accept and value the very many teachers that they will encounter apart from us.

Standard Ebooks: Free and liberated ebooks, carefully produced for the true book lover.

Posted

https://standardebooks.org/

This is a great site. It doesn’t have anything like the number of books to be found on sites like Project Gutenberg or Archive.org, but each one has been lovingly edited and restored to how it should be, and made available in most popular e-book formats. This is a wonderful service and, at a stroke, removes many of the barriers to reading classic (and some not-so-classic) public domain works. It’s all managed through Github. I feel that I should help with this worthy project though I’m not sure I have the time or patience. I’m very grateful to those that do, though.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6177077/standard-ebooks-free-and-liberated-ebooks-carefully-produced-for-the-true-book-lover

This New ‘Ion Drive’ Airplane Flew Straight out of Science Fiction

Posted

https://medium.com/popular-science/this-new-ion-drive-airplane-flew-straight-out-of-science-fiction-4ce0c494a95d

ion powered planeWell, this is cool and, indeed, right out of the pages of science fiction. It uses an array of highly charged electrodes on the wings, positively and negatively charged. The first in line ionizes nitrogen, which is therefore attracted to the second and, in the process, bumps oxygen molecules out of the way providing the driving force. The ‘engine’ is virtually silent, creates zero emissions in flight (the nitrogen ions regain their lost electron at the second electrode) and can be used to direct the plane, in principle, in any direction. The smart folk at MIT who built this flew a test plane indoors for only a couple of hundred metres and, right now, it is highly inefficient compared with the alternatives, but this has got to be the coolest air transport technology ever. It’s a frigging ion drive! Though these are early days and it will be some time before it is safe and efficient enough to be of general use, I love this idea. Lots of other uses, too, like hover-cars, boats that glide over water without touching it (or making noise), super-silent fans, hover-boards, and so on. Throw in some super-efficient solar power and we (or, for us oldies, our descendants) could be sailing on light before too long. I know next to nothing about the physics so maybe these are impossible or, in the case of fans, probably already exist (it’s an obvious technology), but the adjacent possible suddenly got much bigger.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6156561/this-new-%E2%80%98ion-drive%E2%80%99-airplane-flew-straight-out-of-science-fiction

Social physics (collection from Scientific Reports at nature.com)

Posted

https://www.nature.com/collections/hfafjbjbgd

This is an incredibly large (more than 50) and broad collection of recent articles from nature.com – all or almost all of which appear to be open access – that in some way apply methods from physics to social contexts. There’s a predictably large assortment of social networking studies but there’s a lot more to it than that, including quite a bit of pattern matching and discovery relating to many different kinds of set (interestingly enough, perhaps outnumbering the network-based studies), spatial issues, temporal issues, economics studies, collective behaviour analysis, game theory, and cognitive issues. It is going to take me a long time to work my way through these but I think I’m going to learn a lot.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/6097771/social-physics-collection-from-scientific-reports-at-naturecom

A novel approach to protecting academic freedom of speech: allow it, but do not allow it to be heard

Posted

The faculty and professional staff union at Athabasca University, AUFA (the Athabasca University Faculty Association), has two mailing lists, one used for announcements from its exec committee, and one for discussions between its members. Given that most of us have barely any physical contact with one another at the best of times, and that there are no other technologies that are likely to reach even a fraction of all staff involved in teaching and research (the Landing AUFA group, for instance, has only about 40 out of a few hundred potential members) the latter is the primary vehicle through which we, as a community of practice, communicate, share ideas and news, and engage in discussions that help to establish our collective identity. It’s a classic online learning community using a very low threshold, simple, universally accessible technology.

There had been a debate on the discussion list for a few days over the past week on a contentious issue pitting academic freedom against the needs and rights of transgender people. As too often happens when the rights of disadvantaged minorities are involved, the conversation was getting toxic, culminating in a couple of faculty members directly and very unprofessionally abusing another, telling him to shut up and to stop displaying his ignorance. This is not behaviour worthy of anyone, let alone teachers (of all people), and something had to be done about it. At this point the obvious solution would have been for the managers of the list to discuss these abuses individually with those members, and/or for the individuals themselves to reflect on and apologize for their behaviour, and/or to open up the debate on the list about acceptable norms and approaches to de-escalating situations like this. Sadly, that’s not how the list managers responded. Very suddenly, and without any prior warning or discussion whatsoever, the union executive committee shut the entire discussion list down indefinitely, mercilessly nuking it with the following terse and uninformative message posted to the announcement list:

” Dear AUFA members,

Until further notice, AUFA is suspending the AUFA discussions list serv for review of harmful language and due to a high volume of complaints.”

Shocked by this baldly authoritarian response, I immediately sent a strong message of protest, that I tempered with recommendations about what would have been an appropriate approach to managing the problem, and suggestions about ways to move forward with alternative methods and tools in future. I received no reply. One long day later, however, the following message was posted to the announcement list:

” Dear AUFA members,

I want to update you on the situation with the AUFA discussions list serv.

AUFA is committed to protecting Academic Freedom. AUFA is equally committed to protecting Human Rights. AUFA did not make the decision to suspend the list serv lightly. As the entity legally responsible for the listserv, AUFA has an obligation to ensure the safety of its members.

The AUFA executive had a lengthy discussion about the purpose and usefulness of the AUFA listserv and is actively considering alternative methods and forums by which members might communicate with each other in the near future. “

That’s it. That’s the whole message. Clearly they did not discuss this with the people who were actually affected, or with those who had been abusive, and they certainly didn’t talk about it with the rest of us. The message itself is remarkably uninformative, raising far more questions than it answers. It reads to me as ‘you have been naughty children and we have decided to send you to your room to think about it’. But I think they must have been following a different discussion than the one I saw because, though there was certainly some unprofessional nastiness and some unsubtle arguments expressed (that were becoming far more refined as the discussion progressed – that’s how free and open debate is supposed to work), I did not spot any human rights abuses during the discussion, and the only abuse of academic freedom I could see was the decision to shut down the list itself. Removing the possibility of speech altogether is certainly a non-traditional approach to protecting freedom of speech.

Notice, too, that in both messages there is a synecdochal conflation of ‘AUFA’ and ‘the AUFA executive committee’. I’m pretty sure that, as a member of AUFA, I would know whether I had been part of such a decision. That’s a bit like a teacher shutting down an online course because someone was rude, then claiming that the class shut it down. It’s a subtle way of abnegating responsibility, suggesting that some technological entity did something when, in fact, it was done by very real and fully responsible people. AUFA did not do this, and AUFA did not make these decisions. A small group of actual, real human beings did it, all by themselves.

I sent a strongly worded (but respectful) response to that one too.

Who owns this?

I think it is clear that the mailing list is not owned by the union executive committee. They are custodians of it, stewards who run it on the behalf of everyone in the union. Shutting it down denies the members of the union their primary means of connection and debate, including debate about this very issue. The message is quite misleading about the AUFA exec’s responsibilities, too: though they do need to be attentive to illegal behaviours, they are not legally responsible for what other people say on the listserv. In fact, the explicit or implicit legal protections afforded to providers of such services are fundamental to allowing much of the Internet to work at all. This is why there is so much outrage and protest against Trump’s efforts to remove such protections in the US right now. And there are lots of ways of handling the problem, from direct personal communication to public debate to the establishment of rules or a social contract to calling in the police. Going nuclear on the service does not fulfill that responsibility at all; it simply evades it.

It is absolutely fair to claim that list managers do have a responsibility to the union members of helping to maintain a non-abusive, safe, supportive online community. However, shutting down the thing they have an obligation to preserve is not just neglect of that responsibility but the worst and most harmful thing they could possibly do to fulfill it. It is like protecting an endangered animal by shooting it.

Ironically, the final message posted on the now-dead discussion list ended with the line:

“One thing I vowed to myself… is that I would never let anyone stop me from saying what I have to say “

Well, that kept like milk.

I feel incensed, abused, and suddenly incredibly isolated from my university and my colleagues. My sense of loss is tangible and intense. It’s lucky that I do have other channels, like this one, to vent my frustration and to bring this to a broader audience. I hope this message gets to at least a few of those who, like me, are feeling cut off and disempowered and, if they have not done so already, that they loudly voice their concerns to those responsible.

Moving on

Unfortunately, though very low threshold and accessible to all, listservs are not great tools for hosting contentious debates. They are extremely soft technologies which means that, on the positive side, they are extremely flexible and very low threshold, but that therefore a great deal of additional process must be added manually by their participants in order to deal with them: distinguishing threads, choosing which to attend to, tracking conversations, managing archived messages, using appropriate subject lines, to name but a few.

Listservs are poor tools for achieving consensus and poor tools for argument. The push nature of the technology means it can be very intrusive but, equally, the fact that we control our mail filters means that it can be completely shut down and ignored, without other participants having any knowledge that their messages are falling on deaf ears. It’s a technology that allows everyone to shout at the same time so it’s unsurprising that it is fertile ground for misunderstandings, confusion, high emotions, and people who forget that they are talking to other people. The very simplicity that makes them so easy to engage with also makes it easier to forget the humans behind the messages. Unless individuals have taken pains to share things about themselves with their messages, there are not even pictures and profiles to serve as a reminder. Though web archives may be available, they are rarely if ever open for continued dialogue: though, in principle, one could reply to a message from months or years ago, that virtually never happens. This means that people tend rush to get their message across before the list moves on to some other topic, with all the risks that entails. It kind of has to be that way. Because of the push nature of the medium, if conversations were to persist then multiple parallel discussions would rapidly overwhelm everyone’s inbox and attention.

For all these reasons and more, as anyone who has ever tried to do so will be painfully aware, managing a mailing list used for open discussion, especially one (like this) that lacks a clear mandate, contract or terms of engagement, takes a lot of manual effort, a fair bit of ingenuity, and a lot of careful attention. When things get out of hand, those who run the list need to take active, timely, creative measures to defuse them. It’s hard but necessary work, that demands sensitivity, a forgiving nature, a willingness to accept abuse with very little chance of being thanked for your efforts and, often, willingness and availability to work far ouside a normal working day (this, as it happens, is also true of many approaches to online teaching). Unfortunately, no one in our union leadership seems willing or able to take on such management. If that’s the case, the solution is not to shut it down. The solution is to pass it on to someone else who can and will moderate it more caringly, perhaps to put some more resources into managing it and, perhaps, to participatively look into rules, norms, and other tools and procedures that might do the job better.

Moving further on

There are hundreds and maybe thousands of tools and methods that can better (or at least differently) support this kind of debate than a listserv. Even the humble threaded forum at least allows such discussions to be segmented and, for those upset by them, ignored. Some allow for threads or people to be (from an individual’s perspective) muted, and many allow forum owners to close discussions in a particular thread without killing the whole thing. Some go beyond crude threads, allowing richer cross-linking between messages and discussions. Some offer authoring help, like in-line searching of previous messages and direct linking to sources or, simple AI to warn when sentiments appear to run high. Many tools allow for simple tricks like karma points, thumbs up, and other low threshold ways of signalling agreement or disagreement, in a manner that shows collective sentiment without a high commitment or fear of reprisal, and that also signals whether a topic is interesting to the crowd without relying on a deluge of messages to show it. Some offer means to reach decisions, from simple votes to computer supported collaborative argumentation tools. Many allow for profiles and other signals of social presence that make the humans behind the messages more visible and salient. Some (blogs, say, like this one) allow for more focused subscribable discussions on specific themes that are managed and owned by the creator of the original post, and that are not as ephemeral as mailing lists. Some offer other tools like persistent shared bookmarks or filesharing that help to organize resources related to themes of debate. Some have recommender systems that show related posts and thus help to situate discussions, and to support connections back to previous discussions. Many have persistence so that learning is reified and searchable, not lost in a stream of thousands of other emails. Some allow for scheduling and time-limited discussions.

Equally, there are lots of process models for reaching consensus on social norms and acceptable behaviours, as well as ways of dealing with issues when they arise. Skills can be developed in stewardship and moderation so that problems are defused before they become severe, or not arise in the first place thanks to careful specification of ground rules or structuring of the process. There are plenty of books and papers on the subject (this is my favourite, especially now that it is free) that delve into great detail. There are ways of taking an holistic approach that takes into account the larger social ecosystem to (for instance) help to build social capital, use different tools for different functions, and so on.

All of these technologies, including process models, methods, and procedures, come with plentiful gotchas – Faustian bargains and monkeys’ paws that can easily cause more problems than they solve and that will never be ideal for all – so this is not a set of decisions that should be entered into lightly or without extensive consultation, participation, and analysis, and it should always be thought of as an ongoing process, never a finished solution. Clearly, it eventually needs to be done. In the meantime, if a listserv is all we have, then we should at least manage it properly. It is not acceptable to simply nuke the only tool we have, even if it is a weak one.

I do realize that union leadership is an extremely hard and often thankless job and, though I frequently feel very critical of things they do on my behalf,  especially when they adopt an archaic ‘us vs them’ vocabulary, I am thankful they do it. I very seldom voice my adverse opinions because I know they are trying to do their best for everyone, I am certainly not willing to take on the enormous commitments involved myself and, without their hard work and principled actions (regardless of occasions when they actively make things worse) we would, on average, be in a far worse place than we are today. However, the union leadership’s response to this has been outrageously authoritarian, disproportionate, insensitive, and deeply harmful, in direct opposition to everything a union should stand for. If this is a reflection of their values then they do not have either my trust or my support.

 

Postscript

Eventually, after nearly two days, I received a one-line personal reply to my original complaint telling me that the suspension of the list is temporary (this may be news to others in the union who have not been told this: you heard it here first, folks!) and that they will, at some unspecified point, be seeking input from members on communication preferences (not consultation, note, or participation, just input). No timelines were given. I am not satisfied with this.