The physics of social spaces are not like the physics of physical spaces

Image: please respect my privacyOver the last week I peripherally participated in an interesting exchange of views on Twitter between Jesse Stommel and Stephen Downes that raises some fascinating issues about the nature of online social spaces. It started with a plea from Jesse:

“Dear [insert company name], searching every mention of your company and jumping into conversations where you haven’t been tagged or invited is invasive. Stop doing that.”

Stephen took exception to this, pointing out that:

“If I use a company name in a public forum, I expect they will take interest and maybe even reply. It’s a *public* forum. That’s how they work.”

What followed explored some fascinating territory, but the essence of the main arguments are (I skim the nuances), on Jesse’s side, that we have a reasonable expectation of being left alone during a private conversation in any public space and, on Stephen’s side, that there should be no expectation of privacy in a public digital space like Twitter, and that any claims to it tread on extremely dangerous ground.  The central question is thus whether there are such things as private conversations on Twitter.

Stephen’s big concern is that, taken to its logical conclusion, laying claim to privacy on Twitter opens the door for outrages like the Proctorio vs Linkletter case, in which Proctorio claimed that “Mr. Linkletter infringed its copyright, circumvented technological protection measures, and breached confidence” by sharing one of its fully public (though not publicized) YouTube videos with students. YouTube quite closely resembles Twitter in its social structure (though little else), so it is a good analogy. Stephen is, I think rightly, concerned at ‘calling out’ individuals or organizations for invading ‘private’ conversations in public spaces because it implies the unilateral imposition of norms, rules of behaviour, and expectations by one individual or group on another, in a space that neither owns.

Jesse’s counter-arguments are interesting, and subtle. He strongly rejects Stephen’s analogy with the Proctorio case because all he is doing is asserting his right to privacy, not abusing his market position or trying to cause harm. It’s just a request to be let alone, calling on what he sees as norms of politeness, not a demand that this should be enshrined in rules or legislation. He observes that, though Twitter is a public space, it has variegation that emerges because of (often tacit, seldom explicit) ways that many (not all) people use it, which in turn is supported by the ways that Twitter’s algorithms push some kinds of tweet more than others. For this particular case in point, he notes that the algorithm tends to broadcast initial tweets more than it does replies, so what follows in a set of replies could be assumed by its participants to be a less public conversation. In fact, as I understand his argument, Jesse thinks of it as a private conversation in a public space, analogous to having a private conversation in a public park where one might be inadvertently overheard, but it would be rude to deliberately listen in or contribute unless invited. If this were a true analogy then I might support it. But, if it is true, then so are quite a few other things, and that’s where it starts to get interesting.

I’ve been a Twitter user for approaching 15 years now and it has never occurred to me till now that any of my conversations might in any way be construed as private. They are sometimes personal, for sure, but definitely not private. Conversations are soft technologies that are flexible, mutable, and situated, and (without further clues like people quietly conversing in a corner) you need to read them in order to know whether you would be intruding on them, which means that they are simply not private. Without further reasons to assume privacy, it is just a conversation in public between two people to which other people are not invited.

So the crux of Jesse’s argument seems to be the notion that a happenstance of Twitter’s current implementation that makes some tweets less likely to be seen than others, combined with a set of norms relating to that, that may or may not be shared by others, allows one to claim that a conversation is not just personal but private.

The physics of online social spaces

Twitter is, as Stephen says and Jesse agrees, for the most part a completely public space (not counting direct messaging or constraints on tweets to only those you follow/are following) but, as the example of the relative prominence given to initial tweets compared with replies to them amply demonstrates, it does have a structure. It is just one that does not obey anything like the same physics as a physical space. You can achieve a measure of privacy in a public physical space because there has to be proximity in space and time in order to communicate at all, and there are limits to human voice projection, ability to hear, and ability to attend to multiple conversations at once. There are also visual clues that people are talking privately. Though there is variegation in structure, none of those limits apply in Twitter or, for that matter, most online social spaces.

Early in the conversation I chipped in to observe that one of the many differences between private conversations in physical space and Twitter exchanges is that tweets are persistent. They are a little like graffiti left in public spaces that continues to communicate long after the initial intent has passed, and may be happened upon at any time in the future in quite different contexts than those imagined by the graffiti artist. Jesse’s response to that was that there’s a difference between graffiti on a public building in five foot high letters and graffiti on a shady tree or in a tunnel. Again, his point is that there are parts of Twitter where there might be a reasonable expectation of relative privacy, where it would be rude to join the conversation. Though I agree that it is often possible to tell from reading a conversation whether you might be welcome or not (and yes, social norms apply to that), my big problem with Jesse’s argument is that proximity in Twitter-space is not just defined by relative position in a dialogue or likelihood of appearance in a Twitter feed, as he seems to imply.

Beyond its support for conversations between individuals, Twitter embodies two distinct but overlapping social forms: the network and the set. @mentions in Twitter combined with its ‘following’ functionality are the main drivers for the network form. If you follow someone or they mention you then your message becomes proximal to them. That’s a big part of Twitter’s physics, and it has no analogue in physical space. Thus, your conversation is very likely to be overheard by others because you are (metaphorically) standing right next to them and chipping your words in five foot letters in stone where they can and will be found, now and in the future. If you wanted to have a private conversation in a park then you wouldn’t stand less than a metre away from someone that you didn’t want to listen in and shout in their face. But that’s not all.

Hashtags and search terms are the main drivers for the set social form, which at least closely competes with if not exceeds the value of social networks in Twitter. When you use a hashtag or even a distinctive word (say, the name of a company or person) then your message becomes proximal to those who follow that hashtag or who have saved a search for that keyword. So you are not just standing right next to everyone in your social network, but to the potentially much larger social set of people who are interested in keywords that you use in your conversation. Again, you might not intend it, you might not even be able to see them, but you are shouting in their faces.

Maybe you do have a right to privacy in any public space, but that right does not overrule simple physics. You have to know  the physics of that space in order to know what ‘private’ means within it. And the simple physics of Twitter means that ‘next to’ and ‘within hearing distance’ extends to anyone with an interest in you or what you are saying in the sentences you write. If you want different social physics that support privacy, then you need to take your conversation to a different space, because Twitter doesn’t work that way. You can ask for non-interference in a personal conversation, but not for privacy.

Designing better social physics

Retrato cubista del escritor español Ramón Gómez De la Sena por el pintor mexicano Diego RiveraAs it happens, we grappled a lot with issues of context and privacy exactly like this when we designed the social physics of the Landing.  Its social physics are deliberately designed to make precisely those nooks and niches that Jesse wants to find in Twitter.  The Landing starts with discretionary access control for every post and every profile field (we chose to build it using the Elgg framework because of its support for this). Like the much missed (and never hit) Google+ it also allows you to create circles, that are not just useful for following but, more significantly, for limiting access to particular individuals. Again, that came for free with Elgg, though we added some enhancements to forefront it, and to make it usable.

It’s not just about the content, though; it’s about presentation of self (we were influenced in this by Goffman’s dramaturgical analysis). We also therefore built a range of context-switching tools – notably tabbed profiles and pinboards (known internally as ‘sets’) – that allow you to present a completely different facade to different circles, groups, and sets of people. This is not just concerned with showing or hiding different fields and content, but with looking completely different and showing completely different stuff to different people. The public facade of my profile is not the same as the one displayed to my friends and, if I wished, I could present different facades to all the different circles or groups of people I follow or belong to. We’ve still not solved the temporal issue – like most social sites, the fundamental unit of communication is still persistent graffiti. In fact, to a large extent we wanted it that way, because it’s a site for collective learning, and so it has to have a collective memory though, like memories in brains, it would be useful to have short-term memories too. However, simply letting posts expire is not the solution, in part due to the many ways that digital content can be copied and archived but, more importantly, because forgetting is and must be an active process that cannot and should not be automated. My earlier CoFIND system did have a way to deal with that (memories had to be actively maintained by active interest and use by members or, though they would never be fully lost, they would be far less likely to be recalled) but we didn’t make much use of that idea on the Landing, save in isolated pockets, because it would have really irritated the many people or groups that engage intermittently (e.g. in iterations of paced courses).

Unfortunately, most of the Landing’s context-switching features are not even slightly intuitive (especially to those already familiar with the cruder social physics of popular social media) so most are very rarely used. Google+, with its massively simplified version of the same idea, probably failed at least in part for this reason. Such complexity can work, with the right membership. Slashdot, for instance, has an extraordinarily rich and ever-evolving social physics, and it has thrived for about 25 years, but the reasons for its success probably lie at least in part in its tagline ‘News for Nerds’. Its members are not phased by complex interfaces, and it is well-enough designed to work reasonably well if you don’t engage with all the features.

Perhaps a bigger issue, though, is that the richer social physics of both Slashdot and the Landing only work if you happen to be a member. For public posts, like this one, the physics are very much like those of Twitter or Facebook.

For now, the best bet is to use different social spaces for different aspects of your life but, thanks largely to Facebook’s single-minded and highly effective undermining of OpenSocial, there’s not a lot of ways to seamlessly move between them right now while retaining a rich and faceted identity. At least there’s still RSS, which is how come you might be reading this on the Landing (where it is originally posted) or at https://jondron.ca/ (which will automagically then push it to Twitter), but it’s not ideal.

It’s very challenging to design a digital space that is both richly supportive of human social needs and easy to use. The Landing is definitely not the solution, but the underlying idea – that people are richly faceted social beings who interact and present themselves differently to different people at different times –  still makes sense to me. As the conversation between Jesse and Stephen shows, there is a need for support for that more than ever.

Mediaeval Teaching in the Digital Age (slides from my keynote at Oxford Brookes University, May 26, 2021)

 front slide, mediaeval teaching

These are the slides from my keynote today at the Oxford Brookes “Theorizing the Virtual” School of Education Research Conference. As theorizing the virtual is pretty much my thing, I was keen to be a part of this! It was an ungodly hour of the day for me (2am kickoff) but it was worth staying up for. It was a great bunch of attendees who really got into the spirit of the thing and kept me wide awake. I wish I could hang around for the rest of it but, on the bright side, at least I’m up at the right time to see the Super Flower Blood Moon (though it’s looking cloudy, darn it).  In this talk I dwelt on a few of the notable differences between online and in-person teaching. This is the abstract…

Pedagogical methods (ways of teaching) are solutions to problems of helping people to learn, in a context filled with economic, physical, temporal, legal, moral, social, political, technological, and organizational constraints. In mediaeval times books were rare and unaffordable, and experts’ time was precious and limited, so lectures were a pragmatic solution, but they in turn created more problems. Counter-technologies such as classes, classrooms, behavioural rules and norms, courses, terms, curricula, timetables and assignment deadlines were were devised to solve those problems, then methods of teaching (pedagogies) were in turn invented to solve problems these counter-technologies caused, notably including:
· people who might not want (or be able) to be there at that time,
· people who were bored and
· people who were confused.
Better pedagogies supported learner needs for autonomy and competence, or helped learners find relevance to their own goals, values, and interests. They exploited physical closeness for support, role-modelling, inspiration, belongingness and so on. However, increasingly many relied on extrinsic motivators, like classroom discipline, grades and credentials to coerce students to learn. Extrinsic motivation achieves compliance, but it makes the reward or avoidance of the punishment the goal, persistently and often permanently crowding out intrinsic motivation. Intelligent students respond with instrumental approaches, satisficing, or cheating. Learning seldom persists; love of the subject is subdued; learners learn to learn in ineffective ways. More layers of counter-technologies are needed to limit the damage, and so it goes on.
Online, the constraints are very different, and its native forms are the motivational inverse of in-person learning. An online teacher cannot control every moment of a learner’s time, and learners can use the freedoms they gain to take the time they need, when they need it, to learn and to reflect, without the constraints of scheduled classroom hours and deadlines. However, more effort is usually needed to support their needs for relatedness. Unfortunately, many online teachers try (or are required) to re-establish the control they had in the classroom through grading or the promise of credentials, recreating the mediaeval problems that would otherwise not exist, using tools like learning management systems that were designed (poorly) to replicate in-person teaching functions. These are solutions to the problems caused by counter-technologies, not to problems of learning.
There are better ways, and that’s what this session is about.

front slide, mediaeval teaching

Educational technology: what it is and how it works | AI & Society

https://rdcu.be/ch1tl

This is a link to my latest paper in the journal AI & Society. You can read it in a web browser from there, but it is not directly downloadable. A preprint of the submitted version (some small differences and uncorrected errors here and there, notably in citations) can be downloaded from https://auspace.athabascau.ca/handle/2149/3653. The published version should be downloadable for free by Researchgate members.

This is a long paper (about 10,000 words), that summarizes some of the central elements of the theoretical model of learning, teaching and technology developed in my recently submitted book (still awaiting review) and that gives a few examples of its application. For instance, it explains:

  • why, on average researchers find no significant difference between learning with and without tech.
  • why learning styles theories are a) inherently unprovable, b) not important even if they were, and c) a really bad idea in any case.
  • why bad teaching sometimes works (and, conversely, why good teaching sometimes fails)
  • why replication studies cannot be done for most educational interventions (and, for the small subset that are susceptible to reductive study, all you can prove is that your technology works as intended, not whether it does anything useful).

Abstract

This theoretical paper elucidates the nature of educational technology and, in the process, sheds light on a number of phenomena in educational systems, from the no-significant-difference phenomenon to the singular lack of replication in studies of educational technologies.  Its central thesis is that we are not just users of technologies but coparticipants in them. Our participant roles may range from pressing power switches to designing digital learning systems to performing calculations in our heads. Some technologies may demand our participation only in order to enact fixed, predesigned orchestrations correctly. Other technologies leave gaps that we can or must fill with novel orchestrations, that we may perform more or less well. Most are a mix of the two, and the mix varies according to context, participant, and use. This participative orchestration is highly distributed: in educational systems, coparticipants include the learner, the teacher, and many others, from textbook authors to LMS programmers, as well as the tools and methods they use and create.  From this perspective,  all learners and teachers are educational technologists. The technologies of education are seen to be deeply, fundamentally, and irreducibly human, complex, situated and social in their constitution, their form, and their purpose, and as ungeneralizable in their effects as the choice of paintbrush is to the production of great art.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/8692242/my-latest-paper-educational-technology-what-it-is-and-how-it-works

My keynote slides from Confluence 2021 – STEAM engines: on building and testing the machines in our students’ minds

STEAM Engines

These are my slides for my keynote talk at the IEEE 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence-2021), hosted by Amity University, India, 28th January 2021. Technically it was 27th January here in Vancouver when I started, but 28th January when I finished. I hate timezones.

The talk winds up being about how to be a (mainly online) teacher in science, technology, engineering, and mathematics (STEM) – not how to teach, as such – but it gets to the point circuitously through discussing some aspects of the nature of technology, using a subset of my coparticipation model. In (very brief) the idea behind that is that ‘technology’ means organizing stuff to do stuff (any stuff), and we are not just users but participants in that organization, either playing our roles correctly (hard technologies) or organizing stuff ourselves (soft technologies). Almost always, thanks to the fact that almost all technologies are assemblies of and with other technologies, it is a mix of the two. In the technologies of learning there are many coparticipants, all playing roles, soft or hard or both. The designated teacher is only one of these, of varying significance.

The talk dwelt on the technological nature of teaching itself, and on the technological nature of the results of teaching. Teaching (as a distributed process) can usefully be seen as a process of building technologies in learners’ minds, some hard (training), some soft (teaching). These technologies can, like all technologies, be assembled together or with others, so our minds are both enacted and extended through technologies with one another and with the constructed world around us.

In STEM subjects there is a tendency to focus a lot more on building hard technologies than on soft technologies, because there tends to be a lot of hard stuff to learn before you can do anything much at all. There are many other subjects like this, including one of the biggest, language learning. The same is actually true in softer disciplines but students tend to come equipped with a lot of the basic hard stuff – especially language, debating skills, etc – already, so a really big part of the machine already exists. However, as much as it is in the liberal arts (the ‘A’ in STEAM), it is actually the soft technologies – what we do with those hard machines in our minds, the soft technologies we assemble with them – that actually matters, personally, in the workplace, and in our social lives. Also, from a motivational perspective it is normally a really bad idea to force people to learn a lot of hard stuff without them actually having a personal need or desire to do so. Training people in the hard stuff without using it in a soft, personally/socially relevant and meaningful context is a recipe for failure, though the fact that hard skills and knowledge can be accurately measured means that assessments of it tend to create an illusion of success. ‘Success’, though, just means that the hard machine works as intended, not that it actually does anything useful.

Avoiding this chicken and egg problem – the need for hard skills before you can do anything, but the uselessness of them in isolation – is not difficult. In fact, it is how we learn to speak, and many other things. It means letting go of the notion that teachers control everything, embracing the distributed nature of teaching, and designing ways of learning that support autonomy, achievable challenge, and relatedness. To do this means making learning (not just its products) visible, creating a culture and tools for sharing, and designing in support processes to help learners overcome obstacles. Basically, from a designated teacher’s perspective, it’s about letting go and staying close. It’s much the same as how we bring up our kids, as it happens.

It was an odd session, a lecture with no direct interaction. In itself, this would not be a great learning experience for anyone. However – and this is one of my big points – it is the assembly that matters, not the individual components, and I was not the one doing that assembly. Seen as a component of learning, attended without coercion or extrinsic goals, my little lecture is something that can be assembled to make something quite useful.

How distance changes everything: slides from my keynote at the University of Ottawa

These are the slides from my keynote at the University of Ottawa’s “Scaffolding a Transformative Transition to Distance and Online Learning” symposium today. In the presentation I discussed why distance learning really is different from in-person learning, focusing primarily on the fact that they are the motivational inverse of one another. In-person teaching methods evolved in response to the particular constraints and boundaries imposed by physics, and consist of many inventions – pedagogical and otherwise – that are counter-technologies designed to cope with the consequences of teaching in a classroom, a lot of which are not altogether wise. Many of those constraints do not exist online, and yet we continue to do very similar things, especially those that control and dictate what students should do, as well as when, and how they should do it. This makes no sense, and is actually antagonistic to the natural flow of online learning. I provided a few simple ideas and prompts for thinking about how to go more with the flow.

The presentation was only 20 minutes of a lively and inspiring hour-long session, which was fantastic fun and provided me with many interesting questions and a chance to expand further on the ideas.

uottawa2020HowDistanceChangesEverything

Joyful assessment: beyond high-stakes testing

Here are my slides from my presentation at the Innovate Learning Summit yesterday. It’s not world-shattering stuff – just a brutal attack on proctored, unseen written exams (PUWEs, pronounced ‘pooies’), followed by a description of the rationale, process, benefits, and unwanted consequences behind the particular portfolio-based approach to assessment employed in most of my teaching. It includes a set of constraints that I think are important to consider in any assessment process, grouped into pedagogical, motivational, and housekeeping (mainly relating to credentials) clusters. I list 13 benefits of my approach relating to each of those clusters, which I think make a pretty resounding case for using it instead of traditional assignments and tests. However, I also discuss outstanding issues, most of which relate to the external context and expectations of students or the institution, but a couple of which are fairly fundamental flaws (notably the extreme importance of prompt, caring, helpful instructor/tutor engagement in making it all work, which can be highly problematic when it doesn’t happen) that I am still struggling with.

Evaluating assessment

Exam A group of us at AU have begun discussions about how we might transform our assessment practices, in the light of the far-reaching AU Imagine plan and principles. This is a rare and exciting opportunity to bring about radical and positive change in how learning happens at the institution. Hard technologies influence soft more than vice versa, and assessments (particularly when tied to credentials) tend to be among the hardest of all technologies in any pedagogical intervention. They are therefore a powerful lever for change. Equally, and for the same reasons, they are too often the large, slow, structural elements that infest systems to stunt progress and innovation.

Almost all learning must involve assessment, whether it be of one’s own learning, or provided by other people or machines. Even babies constantly assess their own learning. Reflection is assessment. It is completely natural and it only gets weird when we treat it as a summative judgment, especially when we add grades or credentials to the process, thus normally changing the purpose of learning from achieving competence to achieving a reward. At best it distorts learning, making it seem like a chore rather than a delight, at worst it destroys it, even (and perhaps especially) when learners successfully comply with the demands of assessors and get a good grade. Unfortunately, that’s how most educational systems are structured, so the big challenge to all teachers must be to eliminate or at least to massively reduce this deeply pernicious effect. A large number of the pedagogies that we most value are designed to solve problems that are directly caused by credentials. These pedagogies include assessment practices themselves.

With that in mind, before the group’s first meeting I compiled a list of some of the main principles that I adhere to when designing assessments, most of which are designed to reduce or eliminate the structural failings of educational systems. The meeting caused me to reflect a bit more. This is the result:

Principles applying to all assessments

  • The primary purpose of assessment is to help the learner to improve their learning. All assessment should be formative.
  • Assessment without feedback (teacher, peer, machine, self) is judgement, not assessment, pointless.
  • Ideally, feedback should be direct and immediate or, at least, as prompt as possible.
  • Feedback should only ever relate to what has been done, never the doer.
  • No criticism should ever be made without also at least outlining steps that might be taken to improve on it.
  • Grades (with some very rare minor exceptions where the grade is intrinsic to the activity, such as some gaming scenarios or, arguably, objective single-answer quizzes with T/F answers) are not feedback.
  • Assessment should never ever be used to reward or punish particular prior learning behaviours (e.g. use of exams to encourage revision, grades as goals, marks for participation, etc) .
  • Students should be able to choose how, when and on what they are assessed.
  • Where possible, students should participate in the assessment of themselves and others.
  • Assessment should help the teacher to understand the needs, interests, skills, and gaps in knowledge of their students, and should be used to help to improve teaching.
  • Assessment is a way to show learners that we care about their learning.

Specific principles for summative assessments

A secondary (and always secondary) purpose of assessment is to provide evidence for credentials. This is normally described as summative assessment, implying that it assesses a state of accomplishment when learning has ended. That is a completely ridiculous idea. Learning doesn’t end. Human learning is not in any meaningful way like programming a computer or storing stuff in a database. Knowledge and skills are active, ever-transforming, forever actively renewed, reframed, modified, and extended. They are things we do, not things we have.

With that in mind, here are my principles for assessment for credentials (none of which supersede or override any of the above core principles for assessment, which always apply):

  • There should be no assessment task that is not in itself a positive learning activity. Anything else is at best inefficient, at worst punitive/extrinsically rewarding.
  • Assessment for credentials must be fairly applied to all students.
  • Credentials should never be based on comparisons between students (norm-referenced assessment is always, unequivocally, and unredeemably wrong).
  • The criteria for achieving a credential should be clear to the learner and other interested parties (such as employers or other institutions), ideally before it happens, though this should not forestall the achievement and consideration of other valuable outcomes.
  • There is no such thing as failure, only unfinished learning. Credentials should only celebrate success, not punish current inability to succeed.
  • Students should be able to choose when they are ready to be assessed, and should be able to keep trying until they succeed.
  • Credentials should be based on evidence of competence and nothing else.
  • It should be impossible to compromise an assessment by revealing either the assessment or solutions to it.
  • There should be at least two ways to demonstrate competence, ideally more. Students should only have to prove it once (though may do so in many ways and many times, if they wish).
  • More than one person should be involved in judging competence (at least as an option, and/or on a regularly taken sample).
  • Students should have at least some say in how, when, and where they are assessed.
  • Where possible (accepting potential issues with professional accreditation, credit transfer, etc) they should have some say over the competencies that are assessed, in weighting and/or outcome.
  • Grades and marks should be avoided except where mandated elsewhere. Even then, all passes should be treated as an ‘A’ because students should be able to keep trying until they excel.
  • Great success may sometimes be worthy of an award – e.g. a distinction – but such an award should never be treated as a reward.
  • Assessment for credentials should demonstrate the ability to apply learning in an authentic context. There may be many such contexts.
  • Ideally, assessment for credentials should be decoupled from the main teaching process, because of risks of bias, the potential issues of teaching to the test (regardless of individual needs, interests and capabilities) and the dangers to motivation of the assessment crowding out the learning. However, these risks are much lower if all the above principles are taken on board.

I have most likely missed a few important issues, and there is a bit of redundancy in all this, but this is a work in progress. I think it covers the main points.

Further random reflections

There are some overriding principles and implied specifics in all of this. For instance, respect for diversity, accessibility, respect for individuals, and recognition of student control all fall out of or underpin these principles. It implies that we should recognize success, even when it is not the success we expected, so outcome harvesting makes far more sense than measurement of planned outcomes. It implies that failure should only ever be seen as unfinished learning, not as a summative judgment of terminal competence, so appreciative inquiry is far better than negative critique. It implies flexibility in all aspects of the activity. It implies, above and beyond any other purpose, that the focus should always be on learning. If assessment for credentials adversely affects learning then it should be changed at once.

In terms of implementation, while objective quizzes and their cousins can play a useful formative role in helping students to self-assess and to build confidence, machines (whether implemented by computers or rule-following humans) should normally be kept out of credentialling. There’s a place for AI but only when it augments and informs human intelligence, never when it behaves autonomously. Written exams and their ilk should be avoided, unless they conform to or do not conflict with all the above principles: I have found very few examples like this in the real world, though some practical demonstrations of competence in an authentic setting (e.g. lab work and reporting) and some reflective exercises on prior work can be effective.

A portfolio of evidence, including a reflective commentary, is usually going to be the backbone of any fair, humane, effective assessment: something that lets students highlight successes (whether planned or not), that helps them to consolidate what they have learned, and that is flexible enough to demonstrate competence shown in any number of ways. Outputs or observations of authentic activities are going to be important contributors to that. My personal preference in summative assessments is to only use the intended (including student-generated) and/or harvested outcomes for judging success, not for mandated assignments. This gives flexibility, it works for every subject, and it provides unquivocal and precise evidence of success. It’s also often good to talk with students, perhaps formally (e.g. a presentation or oral exam), in order to tease out what they really know and to give instant feedback. It is worth noting that, unlike written exams and their ilk, such methods are actually fun for all concerned, albeit that the pleasure comes from solving problems and overcoming challenges, so it is seldom easy.

Interestingly, there are occasions in traditional academia where these principles are, for the most part, already widely applied. A typical doctoral thesis/dissertation, for example, is often quite close to it (especially in more modern professional forms that put more emphasis on recording the process), as are some student projects. We know that such things are a really good idea, and lead to far richer, more persistent, more fulfilling learning for everyone. We do not do them ubiquitously for reasons of cost and time. It does take a long time to assess something like this well, and it can take more time during the rest of the teaching process thanks to the personalization (real personalization, not the teacher-imposed form popularized by learning analytics aficionados) and extra care that it implies. It is an efficient use of our time, though, because of its active contribution to learning, unlike a great many traditional assessment methods like teacher-set assignments (minimal contribution) and exams (negative contribution).  A lot of the reason for our reticence, though, is the typical university’s schedule and class timetabling, which makes everything pile on at once in an intolerable avalanche of submissions. If we really take autonomy and flexibility on board, it doesn’t have to be that way. If students submit work when it is ready to be submitted, if they are not all working in lock-step, and if it is a work of love rather than compliance, then assessment is often a positively pleasurable task and is naturally staggered. Yes, it probably costs a bit more time in the end (though there are plenty of ways to mitigate that, from peer groups to pedagogical design) but every part of it is dedicated to learning, and the results are much better for everyone.

Some useful further reading

This is a fairly random selection of sources that relate to the principles above in one way or another. I have definitely missed a lot. Sorry for any missing URLs or paywalled articles: you may be able to find downloadable online versions somewhere.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. Retrieved from https://www.jhsph.edu/departments/population-family-and-reproductive-health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-learning.pdf

Boud, D. (2007). Reframing assessment as if learning were important. Retrieved from https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in organizational change and development, 1, 129-169.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3/4), 325-346.

Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3(3), 220-233.

Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes (Kindle ed.). Mariner Books. (this one is worth forking out money for).

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. Retrieved from http://www.alfiekohn.org/blogs/personalized/ (check out Alfie Kohn’s whole site for plentiful other papers and articles – consistently excellent).

Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-Determination research (pp. 183-203). Rochester, NY: The University of Rochester Press.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications. (may be worth paying for if such things interest you).

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Cairo: Ford Foundation. http://www.managingforimpact.org/sites/default/files/resource/outome_harvesting_brief_final_2012-05-2-1.pdf.

BOOK: Teaching Crowds: Learning and Social Media

About the Book

Within the rapidly expanding field of educational technology, learners and educators must confront a seemingly overwhelming selection of tools designed to deliver and facilitate both online and blended learning. Many of these tools assume that learning is configured and delivered in closed contexts, through learning management systems (LMS). However, while traditional “classroom” learning is by no means obsolete, networked learning is in the ascendant. A foundational method in online and blended education, as well as the most common means of informal and self-directed learning, networked learning is rapidly becoming the dominant mode of teaching as well as learning.

In Teaching Crowds, Dron and Anderson introduce a new model for understanding and exploiting the pedagogical potential of Web-based technologies, one that rests on connections — on networks and collectives — rather than on separations. Recognizing that online learning both demands and affords new models of teaching and learning, the authors show how learners can engage with social media platforms to create an unbounded field of emergent connections. These connections empower learners, allowing them to draw from one another’s expertise to formulate and fulfill their own educational goals. In an increasingly networked world, developing such skills will, they argue, better prepare students to become self-directed, lifelong learners.

 

Address of the bookmark: http://www.aupress.ca/index.php/books/120235

Transactional distance and new media literacies

Moore’s theory of transactional distance describes the communications and psychological gulf between learner and teacher in a distance education setting. The theory was formulated in a correspondence era of distance learning and matured in an era where discussion forums and virtual learning environments reduced transactional distance in a closed-group setting that enabled interactions akin to those in a traditional classroom. In recent years the growth of social networking and social interest sites has led to social forms that fit less easily in these traditional formal models of teaching and learning. When the “teacher” is distributed through the network or is an anonymous agent in a set or is an emergent actor formed by collective intelligence, transactional distance becomes a more complex variable. Evolved social literacies are mutated by new social forms and require us to establish new or modified ways of thinking about learning and teaching. In this missive we explore the notion of transactional distance and the kinds of social literacy that are required for or that emerge from network, set, and collective modes of social engagement. We discuss issues such as preferential attachment, confirmation bias, and trust and describe social literacies needed to cope with them.

Address of the bookmark: http://www.mitpressjournals.org/doi/abs/10.1162/IJLM_a_00104#.VEwtAYcfTEI

Agoraphobia and the modern learner

Abstract:Read/write social technologies enable rich pedagogies that centre on sharing and constructing content but have two notable weaknesses. Firstly, beyond the safe, nurturing environment of closed groups, students participating in more or less public network- or set-oriented communities may be insecure in their knowledge and skills, leading to resistance to disclosure. Secondly, it is hard to know who and what to trust in an open environment where others may be equally unskilled or, sometimes, malevolent. We present partial solutions to these problems through the use of collective intelligence, discretionary disclosure controls and mindful design.

Address of the bookmark: http://www-jime.open.ac.uk/jime/article/viewArticle/2014-03/html