A few thoughts on learning management systems, and on integrated learning environments and their implementation

Why do we build digital learning systems to mimic classrooms?

It is understandable that, when we teach in person, we have to occupy and make different uses of the same or similar environments like classrooms, labs, workshops, lecture theatres, and offices. There are huge financial, physical, and organizational constraints on making the environment fit the task, so it would be madness to build a whole new classroom every time we wished to run a different class.

Online, we could build anything we like

But why do we do the same when we teach online? There are countless tools available and, if none are suitable, it is not too hard to build them or modify them to suit our needs. Once they are built, moving between them just takes a tap of a screen or the click of a mouse. Heck, you can even occupy several of them at once if you have a decent monitor or more than one device.

So why don’t we do this?

Here are a few of the more obvious reasons that using the perfect app for the context of study rarely happens:

  • Teachers’ lack of knowledge of the options (it takes time and effort to discover what’s available).
  • Teachers’ lack of skill in using them (most interesting tools have a learning curve, and that gets steeper in inverse proportion to the softness and diversity of the toolset, so most teachers don’t even know how to make the most of what they already have).
  • Lack of time and/or money for development (a real-life application is what it contains, not just the shell that contains it, and it is not always as easy to take existing stuff and put it in a new tool as it might be in a physical space).
  • Costs and difficulties in management (each tool adds costs in managing faults, configuration, accounting for use, performance, and security).
  • Cognitive load involved for learners in adapting to the metaphors, signposts, and methods needed to use the tool itself.

All of these are a direct consequence of the very diversity that would make us want to use different apps in the first place. This is a classic Faustian bargain in which the technology does what we want, and in the process creates new problems to solve.  Every virtual system invents at least some of the dynamics of how people and things interact with it and within it. In effect, every app has its own physics. That makes them harder to find out about, harder to learn, harder to develop, costlier to manage, and more difficult to navigate than the static, fixed facilities found in particular physical locations. They are all different, there are few if any universals, and any universal today may become a conditional tomorrow. Gravity doesn’t necessarily work the same way in virtual systems.

image of a pile of containersAnd so we get learning management systems

The learning management system (LMS) kind of deals with all of these problems: poorly, harmfully, boringly, and painfully, but it does deal with them. Currently, most of the teaching at Athabasca University is through the open source Moodle LMS, lightly modified by us because our needs are not quite like others (self-pacing and all that). But Moodle is not special: in terms of what it does and how it does it, it is not significantly different from any other mainstream LMS – Blackboard, Brightspace, Canvas, Sakai, whatever.

Almost every LMS essentially automates the functions, though not exactly the form, of traditional classrooms. In other parts of the world people prefer to use the term ‘managed learning environment’ (MLE) for such things, and it is the most dominant representative of a larger category of systems usually described as virtual learning environments (VLEs) that also includes things like MOOs (multi-user dungeons, object oriented), immersive learning environments, and simpler web-based teaching systems that replicate aspects of classrooms such as Google Classroom or Microsoft’s gnarly bundle of hastily repurposed rubbish for teaching that I’m not sure even has a name yet. Notice the spatial metaphors in many of these names.

Little boxes made of ticky tacky

The people who originally designed LMSs back in the 90s (I did so myself) based their designs on the functions and entities found in a traditional university because that was their context, and that was where they had to fit. Metaphorically, an LMS or MLE is a big university building with rather uniform classrooms, with perhaps a yard where you can camp out with a few other systems (plugins, LTI hooks, etc) that conform to its requirements and that are allowed in to classrooms when invited, and a few doors and gateways (mainly hyperlinks) linking it circuitously or in jury-rigged fashion to other similarly weakly connected buildings (e.g. places to register, places to seek support, places to talk to an advisor, places to complain, places to find books, and so on). It doesn’t have metaphorical corridors, halls, common rooms, canteens, yards, libraries or any of the other things that normally make up a physical university. You rarely get to even be aware of other classrooms beyond those you are in. Some people (me in a past life) might give classrooms cute names like ‘the learning cafe’ but it’s still just another classroom. You teleport from one classroom to the next because what happens in corridors (really a big lot of incredibly important pedagogically useful stuff, as it happens) is not perceived by the designers as a useful classroom function to be automated or perhaps, more charitably, they just couldn’t figure out how to automate that.

Reified roles

It’s a very controlled environment where everyone has a programmatically enforced role (mostly reflecting traditional educational roles), that may vary according to the room, but that are far less fluid than those in physical spaces. There are strong hierarchies, and limited opportunities for moving between them. Some of those hierarchies are new: the system administrator, for instance, has way more power than anyone in a physical university to determine how learning happens, like an architect with the power to move walls, change the decor, add extensions, and so on, at will. The programmers of the system are almost god-like in their command of its physics. But the ways that they give teachers (or learning designers, or administrators) control, as designers, directors, and regulators of the classroom, are perhaps the most pernicious. In a classroom a teacher may lead (and, by default, usually does). In an LMS, a teacher (or someone playing that role) must lead. The teacher sees things that students cannot, and controls things that the students may not. A teacher configures the space, and determines with some precision how it will be used. With a lot of effort and risk, it can be made to behave differently, but it almost never is.

Functions are everything

An LMS is typically built along functional lines, and those functions are mostly based on loose, superficial observations of what teachers and students seem to do in physical classrooms. The metaphorical classrooms are weird, because they are structured by teaching (seldom learning) function rather than along pedagogical lines: for instance, if you want to talk with someone, you normally need to go to a separate enclosed area inside the classroom or leave a note on the teacher’s desk. Same if you want to take a test, or share your work with others. Another function, another space. Some have many little rooms for different things. Lectures are either literally that (video recordings) or (more usefully, from a learning perspective), text and images to be read on screen, based on the assumption that the only function of lectures is information transmission (it is so very, very much not – that’s its least useful and least effective role). There’s seldom a chance to put even put up your hand to question something. Notices can usually only be pinned on the wall by teachers. Classroom timetables are embodied in software because of course you need a rigid and unforgiving timetable in a medium that sells itself on enabling learning anywhere, any time. Some, including Moodle, will allow you to break up the content differently, but it’s still another timetable; just a timetable without dates. It’s still the teacher who sets the order, pacing and content.

Robot overlords

It’s a high-tech classroom. There are often robots there that are programmed to make you behave in ways determined by those higher in the hierarchy (sometimes teachers, sometimes administrators, sometimes the programmers of the software). For instance, they might act as gatekeepers that prevent you from moving on to the next section before completing the current one, or they might prevent you submitting work before or after a specified date. They might mark your work. There are surveillance cameras everywhere, recording your every move, often only accessible to those with more powerful roles (though sometimes a robot or two might give you a filtered view of it).

Beginnings and ends

You can’t usually go back and visit when your course is over because someone decided it would be a good idea to set opening and closing enrolment dates and assumed that, when they were done, the learning was done (which of course it never is – it keeps on evolving long after explicit teaching and testing occurred). Again, it’s because physical classes are scheduled and terms come to an end because they must be, not because it makes pedagogical sense. And, like almost everything, you can override this default, but hardly anyone ever does, because it brings back those Faustian bargains, especially in manageability.

Dull caricatures of physical spaces

Basically, the LMS is an automated set of metaphorical classrooms that hardens many of the undesirable by-products of educational systems in software in brain-dead ways that have little to do with how best to teach, and that stretch the spatial metaphors that inform it beyond breaking point. Each bit of automation and each navigational decision hardens pedagogical choices. For all the cozy metaphors, programmers invent rather than replicate physics, in the process warping reality in ways that do no good and much harm. Classrooms solved problems of physics for in-person teaching and form part of a much larger structure that has evolved to teach reasonably well (including corridors, common rooms, canteens, and libraries, as it happens). Their more visible functions are only a part of that and, arguably, not the main part. There is much pedagogy embedded in the ways that physical universities, whether by accident or design, have evolved over centuries to support learning in every quadrangle and nook of a coffee shop. LMSs just focus on a limited subset of teaching roles, and empower the teacher in ways that caricature their already excessive dominance in the classroom (which only occurred because it had to, thanks to physics and the constraints it imposed).

LMSs are crap, but they contain recognizable semblances of their physical counterparts and just enough configurability and flexibility to more or less work as teaching tools, a bit, for everyone, almost no matter what their level of digital proficiency might be. They more or less solve the Faustian bargains listed earlier, but they do so by stifling what we wanted and should have been able to do in the first place with online tools, in the process creating new and quite horrific problems, as well as demolishing most of what makes physical universities work in the first place. It never has been true that virtual learning environments are learning environments – they are only ever parts of them – and there are places to escape from them, such as the Landing, other virtual systems, or even just plain old email, but then all those Faustian bargains come back to haunt us again. There has to be a better way.

Beyond the LMS

Cognisant of the issues, Athabasca University is now some way down the path to developing its own distinctive solutions to these problems, in a multi-year multi-million-dollar initiative known as (following the spatial metaphor) the Integrated Learning Environment (ILE). The ILE is not an application. It is an umbrella term for a lot of different, usually independent systems working together as one. Though some of the most interesting opportunities are still only loosely imagined, perhaps because they cause problems that are fiendishly hard to solve (e.g. how can we integrate systems that we build ourselves without creating risks for the rest of the ILE, and what happens when they need to be maintained?) a lot of progress is being made on the non-teaching foundations on which the rest depends (student admin systems, support tools, procedures, etc), as well as on the most visible and perhaps the biggest of its parts, BrightSpace, a proprietary commercial LMS that is meant to replace Moodle, for no obvious pedagogical or technical reasons (it’s no better). It might make economic sense. I don’t know, but I do know that open source software typically costs a fair bit to own, albeit because of the things that make it a much better idea (freedom, flexibility, ownership, etc). There is probably a fair bit of time and money being spent with Desire2Learn (makers of Brightspace) on the things that we spent a fair bit of time and money on many years ago to make Moodle a bit less classroom-like. The choice no doubt has something to do with how reliably and easily it can be made to work with some of the other proprietary commercial systems that someone has decided will make up the ILE. It bothers me greatly that we are not trying hard to choose open source solutions, for reasons that will become clearer in the rest of this post. However, (pedagogically speaking) all the mainstream LMSs are much of a muchness, making the same mistakes as one another in very similar ways, so it probably won’t wreck too much of what we already do within Moodle. But, on its own, it won’t move us much further forward and we could do it better. That’s what the ILE is supposed to do – to make the LMS just a part of a much larger teaching environment, intimately connected with the rest of what the university does for or with students, and extensible with new and better ways of learning, teaching, and assessing learning.

picture of lego bricksLego bricks make poor metaphors

When we were first imagining the ILE, though the approach was admirably participative, engaging much of the university community, I was very worried by the things we were encouraged to focus on. It was all about the functionality, the usability, the design, the tools, the pedagogies, the business systems that supported them. Those things matter, for sure, and should be not be ignored, but they should and will change and grow all the time: in fact, part of the point of building this thing is to do just that. Using the city metaphor, pretty much all that we (collectively) considered were the spaces (the rooms, mainly), and the stuff that goes on inside them, much like LMS designers thought of universities as just collections of classrooms in which teaching functions were performed. Space and stuff are, not uncoincidentally, exactly what Stewart Brand identified long ago as inevitably being the fastest-changing, most volatile parts of any town or city (after site, structure, skin, and services). I’ve written a fair bit on the universality of this principle across all systems. It’s a solid structural principle that applies as much to ecosystems and educational systems as to cities. As Brand observes himself, drawing from O’Neill et al (1986), the larger, slower-changing elements of any system affect the smaller, faster-changing more than vice versa. This is for much the same reasons that path dependencies set in. It’s about the prior providing the context for what follows. Flexible things have to fit into the gaps left by less flexible, older, pre-existing things. In physical spaces, of course these tend to be bigger and/or slower, but the same is true in virtual spaces, where size seldom matters that much, but hardness (inflexibility, brittleness) really does. Though lip service was paid to the word ‘integrated’ in our discussions,  I had the strong feeling that the kind of integration we had in mind was that of a Lego set. In fact, I think we were aiming to find a ‘Lego Athabasca University’ set, with assembly instructions and a picture on the box. The vendors who came to talk with us made much of how effectively they could do that, rather than how effectively they could make it possible for others to do that.

Metaphors matter. Lego bricks have to fit together tightly, in pre-specified ways, especially if you are following a plan. If you want to move them around, you have to dismantle a bit of the structure to fit them in. It’s difficult to integrate things that are not bricks, or that are made by different toy companies to work in different ways. At best you get what Brand calls ‘magazine architecture’, or ‘no road’ architecture, beautiful, fit for purpose, intricate and solid, but slow to learn. Lego is not a terrible way to build, compared with buying everything pre-assembled, but it could be improved.

Signals and boundaries

Drawing inspiration from John Holland’s brilliant last work, Signals & Boundaries, I tried to make the case that, instead, we should be focusing on the boundaries (the interfaces between the buildings and the rest of the city), and the signals that pass between them (the people, the messages, etc, the forms they take and how they move around). In Brand’s terms, I wanted us to be thinking about skin and services, and perhaps even structure, though site – Athabasca University – was a given. Though a few people nodded in agreement, I think it mainly fell on deaf ears. We wanted oven-ready solutions, not the infrastructure to enable those solutions. Though the city metaphor works well, because we are talking about human constructions, others would result in similar ways of thinking: cells in bodies, organisms in ecosystems, brains, termite mounds, and so on. All are organized by boundaries (at many levels of hierarchy) and the signals that pass between them.

The Lego set metaphor – whether deliberately or not – seems to have prevailed for now. A lot of old buildings are being slated for demolition and a lot of new virtual buildings are now being erected as part of this development, many of them chosen not because of problems with existing buildings but so that they can more easily connect together and live in the same cloud. This will very likely work, for now, but it is not cheap and it is not flexible, especially given the fact that most of it is not open so, like a rental property, we are not allowed to fix things, add utilities, change the walls, etc, and we are wholly dependent on the landlords being nice to us and each other (knowing that some – ahem, Microsoft – have a long history of abusing their tenants). Those buildings will age. We will find them cramped. Some will age faster than others, and will have to be modified to keep up, perhaps at high cost. Companies renting them might go out of business or change their terms so we might have to demolish the buildings and rent/make new ones. We will be annoyed at how they do things, usually without asking us. We will hate the landlords who dictate what we can do and how we can do it, and who will keep upping the rent while not doing what we ask. We will want more, and the only way to get it will be to build extensions, buy new brick sets, if it is not enough to pay someone to remodel the interiors (and it won’t be). Of course, because most of the big structural elements will not be open source, we will not be able to do that ourselves.

What the ILE really should be

The ILE is, I think, poorly named, because it should not be an environment at all. Following the building metaphor, the ILE is (or should be) more like the system that connects a lot of buildings, bringing them together into a coherent, safe, livable community. It’s infrastructure and services; it is the roads, the traffic signals, the doors, the sidewalks, the water pipes, the waste pipes, the electricity, the network cables; it is the services – fire, police, schools, traffic control, etc; it is all the many rules, standards, norms and regulations that make them work together to help make an environment in which people can live, work, play, and grow. It’s part of the environment – the part that makes it work – but it is not the environment itself. The environment itself is Athabasca University, not just the tools, processes, and systems that support its functions. That includes, most importantly, the people who are part of the university, or who are visitors to it, who are not just users of the environment or dwellers in its walls, but who are or should be the most significant and visible parts of it, just as trees are part of the environment of forests, not users of the forest. Those people live in physical as well as other virtual environments (social media, Word documents, websites, etc) that the ILE can connect together too, to make them a part of it, so the spatial metaphor gets weird at this point. The ILE makes environmental boundaries fuzzy, permeable, and shifting. It’s not an ILE, it’s an ILI – an integrated learning infrastructure.

If we focused on the connections and interfaces, and on how information and processes need to pass across them, and if we thought hard about the nature of those signals, then we could build a system that is resilient, that adapts, that lasts, that grows, that evolves, with parts that we can seamless replace or improve because the interfaces – the building facades, the mains pipes, the junction boxes, etc – will mostly stay the same, evolving slowly as they should. This is about strategy, not planning,  a way of thinking about systems rather than a sequence of things to do.

Some of the key people involved in the process realize this. They are talking about standards, protocols, and projects to build interfaces between systems, and imagining future needs, though they are inevitably distracted by the process of renting Lego bricks, so I am not sure how much they will be able to stay focused on that. I hope they prevail over those who think they are building a set of classrooms and tightly connected admin offices out of self-contained interlocking bricks because our future depends on getting it right. We are aiming to grow. It just takes one critical piece in the Lego building to fail to support that, and the rest falls apart like a… well, like a pile of bricks.

References

Brand, S. (1997). How buildings learn. Phoenix Illustrated. https://www.penguinrandomhouse.ca/books/320919/how-buildings-learn-by-stewart-brand/9780140139969

Holland, J. H. (2012). Signals and Boundaries: Building Blocks for Complex Adaptive Systems. MIT Press.  https://mitpress.mit.edu/books/signals-and-boundaries

O’Neill, R.V., DeAngelis, D.L, Waide, J. B., & Allen, T. F. H. (1986). A Hierarchical Concept of Ecosystems. Princeton University Press. http://www.gbv.de/dms/bs/toc/025157787.pdf

Postman, N. (1998). Five things we need to know about technological change. Denver, Colorado, 28.  https://student.cs.uwaterloo.ca/~cs492/papers/neil-postman–five-things.html

The physics of social spaces are not like the physics of physical spaces

Image: please respect my privacyOver the last week I peripherally participated in an interesting exchange of views on Twitter between Jesse Stommel and Stephen Downes that raises some fascinating issues about the nature of online social spaces. It started with a plea from Jesse:

“Dear [insert company name], searching every mention of your company and jumping into conversations where you haven’t been tagged or invited is invasive. Stop doing that.”

Stephen took exception to this, pointing out that:

“If I use a company name in a public forum, I expect they will take interest and maybe even reply. It’s a *public* forum. That’s how they work.”

What followed explored some fascinating territory, but the essence of the main arguments are (I skim the nuances), on Jesse’s side, that we have a reasonable expectation of being left alone during a private conversation in any public space and, on Stephen’s side, that there should be no expectation of privacy in a public digital space like Twitter, and that any claims to it tread on extremely dangerous ground.  The central question is thus whether there are such things as private conversations on Twitter.

Stephen’s big concern is that, taken to its logical conclusion, laying claim to privacy on Twitter opens the door for outrages like the Proctorio vs Linkletter case, in which Proctorio claimed that “Mr. Linkletter infringed its copyright, circumvented technological protection measures, and breached confidence” by sharing one of its fully public (though not publicized) YouTube videos with students. YouTube quite closely resembles Twitter in its social structure (though little else), so it is a good analogy. Stephen is, I think rightly, concerned at ‘calling out’ individuals or organizations for invading ‘private’ conversations in public spaces because it implies the unilateral imposition of norms, rules of behaviour, and expectations by one individual or group on another, in a space that neither owns.

Jesse’s counter-arguments are interesting, and subtle. He strongly rejects Stephen’s analogy with the Proctorio case because all he is doing is asserting his right to privacy, not abusing his market position or trying to cause harm. It’s just a request to be let alone, calling on what he sees as norms of politeness, not a demand that this should be enshrined in rules or legislation. He observes that, though Twitter is a public space, it has variegation that emerges because of (often tacit, seldom explicit) ways that many (not all) people use it, which in turn is supported by the ways that Twitter’s algorithms push some kinds of tweet more than others. For this particular case in point, he notes that the algorithm tends to broadcast initial tweets more than it does replies, so what follows in a set of replies could be assumed by its participants to be a less public conversation. In fact, as I understand his argument, Jesse thinks of it as a private conversation in a public space, analogous to having a private conversation in a public park where one might be inadvertently overheard, but it would be rude to deliberately listen in or contribute unless invited. If this were a true analogy then I might support it. But, if it is true, then so are quite a few other things, and that’s where it starts to get interesting.

I’ve been a Twitter user for approaching 15 years now and it has never occurred to me till now that any of my conversations might in any way be construed as private. They are sometimes personal, for sure, but definitely not private. Conversations are soft technologies that are flexible, mutable, and situated, and (without further clues like people quietly conversing in a corner) you need to read them in order to know whether you would be intruding on them, which means that they are simply not private. Without further reasons to assume privacy, it is just a conversation in public between two people to which other people are not invited.

So the crux of Jesse’s argument seems to be the notion that a happenstance of Twitter’s current implementation that makes some tweets less likely to be seen than others, combined with a set of norms relating to that, that may or may not be shared by others, allows one to claim that a conversation is not just personal but private.

The physics of online social spaces

Twitter is, as Stephen says and Jesse agrees, for the most part a completely public space (not counting direct messaging or constraints on tweets to only those you follow/are following) but, as the example of the relative prominence given to initial tweets compared with replies to them amply demonstrates, it does have a structure. It is just one that does not obey anything like the same physics as a physical space. You can achieve a measure of privacy in a public physical space because there has to be proximity in space and time in order to communicate at all, and there are limits to human voice projection, ability to hear, and ability to attend to multiple conversations at once. There are also visual clues that people are talking privately. Though there is variegation in structure, none of those limits apply in Twitter or, for that matter, most online social spaces.

Early in the conversation I chipped in to observe that one of the many differences between private conversations in physical space and Twitter exchanges is that tweets are persistent. They are a little like graffiti left in public spaces that continues to communicate long after the initial intent has passed, and may be happened upon at any time in the future in quite different contexts than those imagined by the graffiti artist. Jesse’s response to that was that there’s a difference between graffiti on a public building in five foot high letters and graffiti on a shady tree or in a tunnel. Again, his point is that there are parts of Twitter where there might be a reasonable expectation of relative privacy, where it would be rude to join the conversation. Though I agree that it is often possible to tell from reading a conversation whether you might be welcome or not (and yes, social norms apply to that), my big problem with Jesse’s argument is that proximity in Twitter-space is not just defined by relative position in a dialogue or likelihood of appearance in a Twitter feed, as he seems to imply.

Beyond its support for conversations between individuals, Twitter embodies two distinct but overlapping social forms: the network and the set. @mentions in Twitter combined with its ‘following’ functionality are the main drivers for the network form. If you follow someone or they mention you then your message becomes proximal to them. That’s a big part of Twitter’s physics, and it has no analogue in physical space. Thus, your conversation is very likely to be overheard by others because you are (metaphorically) standing right next to them and chipping your words in five foot letters in stone where they can and will be found, now and in the future. If you wanted to have a private conversation in a park then you wouldn’t stand less than a metre away from someone that you didn’t want to listen in and shout in their face. But that’s not all.

Hashtags and search terms are the main drivers for the set social form, which at least closely competes with if not exceeds the value of social networks in Twitter. When you use a hashtag or even a distinctive word (say, the name of a company or person) then your message becomes proximal to those who follow that hashtag or who have saved a search for that keyword. So you are not just standing right next to everyone in your social network, but to the potentially much larger social set of people who are interested in keywords that you use in your conversation. Again, you might not intend it, you might not even be able to see them, but you are shouting in their faces.

Maybe you do have a right to privacy in any public space, but that right does not overrule simple physics. You have to know  the physics of that space in order to know what ‘private’ means within it. And the simple physics of Twitter means that ‘next to’ and ‘within hearing distance’ extends to anyone with an interest in you or what you are saying in the sentences you write. If you want different social physics that support privacy, then you need to take your conversation to a different space, because Twitter doesn’t work that way. You can ask for non-interference in a personal conversation, but not for privacy.

Designing better social physics

Retrato cubista del escritor español Ramón Gómez De la Sena por el pintor mexicano Diego RiveraAs it happens, we grappled a lot with issues of context and privacy exactly like this when we designed the social physics of the Landing.  Its social physics are deliberately designed to make precisely those nooks and niches that Jesse wants to find in Twitter.  The Landing starts with discretionary access control for every post and every profile field (we chose to build it using the Elgg framework because of its support for this). Like the much missed (and never hit) Google+ it also allows you to create circles, that are not just useful for following but, more significantly, for limiting access to particular individuals. Again, that came for free with Elgg, though we added some enhancements to forefront it, and to make it usable.

It’s not just about the content, though; it’s about presentation of self (we were influenced in this by Goffman’s dramaturgical analysis). We also therefore built a range of context-switching tools – notably tabbed profiles and pinboards (known internally as ‘sets’) – that allow you to present a completely different facade to different circles, groups, and sets of people. This is not just concerned with showing or hiding different fields and content, but with looking completely different and showing completely different stuff to different people. The public facade of my profile is not the same as the one displayed to my friends and, if I wished, I could present different facades to all the different circles or groups of people I follow or belong to. We’ve still not solved the temporal issue – like most social sites, the fundamental unit of communication is still persistent graffiti. In fact, to a large extent we wanted it that way, because it’s a site for collective learning, and so it has to have a collective memory though, like memories in brains, it would be useful to have short-term memories too. However, simply letting posts expire is not the solution, in part due to the many ways that digital content can be copied and archived but, more importantly, because forgetting is and must be an active process that cannot and should not be automated. My earlier CoFIND system did have a way to deal with that (memories had to be actively maintained by active interest and use by members or, though they would never be fully lost, they would be far less likely to be recalled) but we didn’t make much use of that idea on the Landing, save in isolated pockets, because it would have really irritated the many people or groups that engage intermittently (e.g. in iterations of paced courses).

Unfortunately, most of the Landing’s context-switching features are not even slightly intuitive (especially to those already familiar with the cruder social physics of popular social media) so most are very rarely used. Google+, with its massively simplified version of the same idea, probably failed at least in part for this reason. Such complexity can work, with the right membership. Slashdot, for instance, has an extraordinarily rich and ever-evolving social physics, and it has thrived for about 25 years, but the reasons for its success probably lie at least in part in its tagline ‘News for Nerds’. Its members are not phased by complex interfaces, and it is well-enough designed to work reasonably well if you don’t engage with all the features.

Perhaps a bigger issue, though, is that the richer social physics of both Slashdot and the Landing only work if you happen to be a member. For public posts, like this one, the physics are very much like those of Twitter or Facebook.

For now, the best bet is to use different social spaces for different aspects of your life but, thanks largely to Facebook’s single-minded and highly effective undermining of OpenSocial, there’s not a lot of ways to seamlessly move between them right now while retaining a rich and faceted identity. At least there’s still RSS, which is how come you might be reading this on the Landing (where it is originally posted) or at https://jondron.ca/ (which will automagically then push it to Twitter), but it’s not ideal.

It’s very challenging to design a digital space that is both richly supportive of human social needs and easy to use. The Landing is definitely not the solution, but the underlying idea – that people are richly faceted social beings who interact and present themselves differently to different people at different times –  still makes sense to me. As the conversation between Jesse and Stephen shows, there is a need for support for that more than ever.

Mediaeval Teaching in the Digital Age (slides from my keynote at Oxford Brookes University, May 26, 2021)

 front slide, mediaeval teaching

These are the slides from my keynote today at the Oxford Brookes “Theorizing the Virtual” School of Education Research Conference. As theorizing the virtual is pretty much my thing, I was keen to be a part of this! It was an ungodly hour of the day for me (2am kickoff) but it was worth staying up for. It was a great bunch of attendees who really got into the spirit of the thing and kept me wide awake. I wish I could hang around for the rest of it but, on the bright side, at least I’m up at the right time to see the Super Flower Blood Moon (though it’s looking cloudy, darn it).  In this talk I dwelt on a few of the notable differences between online and in-person teaching. This is the abstract…

Pedagogical methods (ways of teaching) are solutions to problems of helping people to learn, in a context filled with economic, physical, temporal, legal, moral, social, political, technological, and organizational constraints. In mediaeval times books were rare and unaffordable, and experts’ time was precious and limited, so lectures were a pragmatic solution, but they in turn created more problems. Counter-technologies such as classes, classrooms, behavioural rules and norms, courses, terms, curricula, timetables and assignment deadlines were were devised to solve those problems, then methods of teaching (pedagogies) were in turn invented to solve problems these counter-technologies caused, notably including:
· people who might not want (or be able) to be there at that time,
· people who were bored and
· people who were confused.
Better pedagogies supported learner needs for autonomy and competence, or helped learners find relevance to their own goals, values, and interests. They exploited physical closeness for support, role-modelling, inspiration, belongingness and so on. However, increasingly many relied on extrinsic motivators, like classroom discipline, grades and credentials to coerce students to learn. Extrinsic motivation achieves compliance, but it makes the reward or avoidance of the punishment the goal, persistently and often permanently crowding out intrinsic motivation. Intelligent students respond with instrumental approaches, satisficing, or cheating. Learning seldom persists; love of the subject is subdued; learners learn to learn in ineffective ways. More layers of counter-technologies are needed to limit the damage, and so it goes on.
Online, the constraints are very different, and its native forms are the motivational inverse of in-person learning. An online teacher cannot control every moment of a learner’s time, and learners can use the freedoms they gain to take the time they need, when they need it, to learn and to reflect, without the constraints of scheduled classroom hours and deadlines. However, more effort is usually needed to support their needs for relatedness. Unfortunately, many online teachers try (or are required) to re-establish the control they had in the classroom through grading or the promise of credentials, recreating the mediaeval problems that would otherwise not exist, using tools like learning management systems that were designed (poorly) to replicate in-person teaching functions. These are solutions to the problems caused by counter-technologies, not to problems of learning.
There are better ways, and that’s what this session is about.

front slide, mediaeval teaching

Educational technology: what it is and how it works

https://rdcu.be/ch1tl

This is a link to my latest paper in the journal AI & Society. You can read it in a web browser from there, but it is not directly downloadable. A preprint of the submitted version (some small differences and uncorrected errors here and there, notably in citations) can be downloaded from https://auspace.athabascau.ca/handle/2149/3653. The published version should be downloadable for free by Researchgate members.

This is a long paper (about 10,000 words), that summarizes some of the central elements of the theoretical model of learning, teaching and technology developed in my recently submitted book (still awaiting review) and that gives a few examples of its application. For instance, it explains:

  • why, on average researchers find no significant difference between learning with and without tech.
  • why learning styles theories are a) inherently unprovable, b) not important even if they were, and c) a really bad idea in any case.
  • why bad teaching sometimes works (and, conversely, why good teaching sometimes fails)
  • why replication studies cannot be done for most educational interventions (and, for the small subset that are susceptible to reductive study, all you can prove is that your technology works as intended, not whether it does anything useful).

Abstract

This theoretical paper elucidates the nature of educational technology and, in the process, sheds light on a number of phenomena in educational systems, from the no-significant-difference phenomenon to the singular lack of replication in studies of educational technologies.  Its central thesis is that we are not just users of technologies but coparticipants in them. Our participant roles may range from pressing power switches to designing digital learning systems to performing calculations in our heads. Some technologies may demand our participation only in order to enact fixed, predesigned orchestrations correctly. Other technologies leave gaps that we can or must fill with novel orchestrations, that we may perform more or less well. Most are a mix of the two, and the mix varies according to context, participant, and use. This participative orchestration is highly distributed: in educational systems, coparticipants include the learner, the teacher, and many others, from textbook authors to LMS programmers, as well as the tools and methods they use and create.  From this perspective,  all learners and teachers are educational technologists. The technologies of education are seen to be deeply, fundamentally, and irreducibly human, complex, situated and social in their constitution, their form, and their purpose, and as ungeneralizable in their effects as the choice of paintbrush is to the production of great art.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/8692242/my-latest-paper-educational-technology-what-it-is-and-how-it-works

My keynote slides from Confluence 2021 – STEAM engines: on building and testing the machines in our students’ minds

STEAM Engines

These are my slides for my keynote talk at the IEEE 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence-2021), hosted by Amity University, India, 28th January 2021. Technically it was 27th January here in Vancouver when I started, but 28th January when I finished. I hate timezones.

The talk winds up being about how to be a (mainly online) teacher in science, technology, engineering, and mathematics (STEM) – not how to teach, as such – but it gets to the point circuitously through discussing some aspects of the nature of technology, using a subset of my coparticipation model. In (very brief) the idea behind that is that ‘technology’ means organizing stuff to do stuff (any stuff), and we are not just users but participants in that organization, either playing our roles correctly (hard technologies) or organizing stuff ourselves (soft technologies). Almost always, thanks to the fact that almost all technologies are assemblies of and with other technologies, it is a mix of the two. In the technologies of learning there are many coparticipants, all playing roles, soft or hard or both. The designated teacher is only one of these, of varying significance.

The talk dwelt on the technological nature of teaching itself, and on the technological nature of the results of teaching. Teaching (as a distributed process) can usefully be seen as a process of building technologies in learners’ minds, some hard (training), some soft (teaching). These technologies can, like all technologies, be assembled together or with others, so our minds are both enacted and extended through technologies with one another and with the constructed world around us.

In STEM subjects there is a tendency to focus a lot more on building hard technologies than on soft technologies, because there tends to be a lot of hard stuff to learn before you can do anything much at all. There are many other subjects like this, including one of the biggest, language learning. The same is actually true in softer disciplines but students tend to come equipped with a lot of the basic hard stuff – especially language, debating skills, etc – already, so a really big part of the machine already exists. However, as much as it is in the liberal arts (the ‘A’ in STEAM), it is actually the soft technologies – what we do with those hard machines in our minds, the soft technologies we assemble with them – that actually matters, personally, in the workplace, and in our social lives. Also, from a motivational perspective it is normally a really bad idea to force people to learn a lot of hard stuff without them actually having a personal need or desire to do so. Training people in the hard stuff without using it in a soft, personally/socially relevant and meaningful context is a recipe for failure, though the fact that hard skills and knowledge can be accurately measured means that assessments of it tend to create an illusion of success. ‘Success’, though, just means that the hard machine works as intended, not that it actually does anything useful.

Avoiding this chicken and egg problem – the need for hard skills before you can do anything, but the uselessness of them in isolation – is not difficult. In fact, it is how we learn to speak, and many other things. It means letting go of the notion that teachers control everything, embracing the distributed nature of teaching, and designing ways of learning that support autonomy, achievable challenge, and relatedness. To do this means making learning (not just its products) visible, creating a culture and tools for sharing, and designing in support processes to help learners overcome obstacles. Basically, from a designated teacher’s perspective, it’s about letting go and staying close. It’s much the same as how we bring up our kids, as it happens.

It was an odd session, a lecture with no direct interaction. In itself, this would not be a great learning experience for anyone. However – and this is one of my big points – it is the assembly that matters, not the individual components, and I was not the one doing that assembly. Seen as a component of learning, attended without coercion or extrinsic goals, my little lecture is something that can be assembled to make something quite useful.

How distance changes everything: slides from my keynote at the University of Ottawa

These are the slides from my keynote at the University of Ottawa’s “Scaffolding a Transformative Transition to Distance and Online Learning” symposium today. In the presentation I discussed why distance learning really is different from in-person learning, focusing primarily on the fact that they are the motivational inverse of one another. In-person teaching methods evolved in response to the particular constraints and boundaries imposed by physics, and consist of many inventions – pedagogical and otherwise – that are counter-technologies designed to cope with the consequences of teaching in a classroom, a lot of which are not altogether wise. Many of those constraints do not exist online, and yet we continue to do very similar things, especially those that control and dictate what students should do, as well as when, and how they should do it. This makes no sense, and is actually antagonistic to the natural flow of online learning. I provided a few simple ideas and prompts for thinking about how to go more with the flow.

The presentation was only 20 minutes of a lively and inspiring hour-long session, which was fantastic fun and provided me with many interesting questions and a chance to expand further on the ideas.

uottawa2020HowDistanceChangesEverything

Joyful assessment: beyond high-stakes testing

Here are my slides from my presentation at the Innovate Learning Summit yesterday. It’s not world-shattering stuff – just a brutal attack on proctored, unseen written exams (PUWEs, pronounced ‘pooies’), followed by a description of the rationale, process, benefits, and unwanted consequences behind the particular portfolio-based approach to assessment employed in most of my teaching. It includes a set of constraints that I think are important to consider in any assessment process, grouped into pedagogical, motivational, and housekeeping (mainly relating to credentials) clusters. I list 13 benefits of my approach relating to each of those clusters, which I think make a pretty resounding case for using it instead of traditional assignments and tests. However, I also discuss outstanding issues, most of which relate to the external context and expectations of students or the institution, but a couple of which are fairly fundamental flaws (notably the extreme importance of prompt, caring, helpful instructor/tutor engagement in making it all work, which can be highly problematic when it doesn’t happen) that I am still struggling with.

Evaluating assessment

Exam A group of us at AU have begun discussions about how we might transform our assessment practices, in the light of the far-reaching AU Imagine plan and principles. This is a rare and exciting opportunity to bring about radical and positive change in how learning happens at the institution. Hard technologies influence soft more than vice versa, and assessments (particularly when tied to credentials) tend to be among the hardest of all technologies in any pedagogical intervention. They are therefore a powerful lever for change. Equally, and for the same reasons, they are too often the large, slow, structural elements that infest systems to stunt progress and innovation.

Almost all learning must involve assessment, whether it be of one’s own learning, or provided by other people or machines. Even babies constantly assess their own learning. Reflection is assessment. It is completely natural and it only gets weird when we treat it as a summative judgment, especially when we add grades or credentials to the process, thus normally changing the purpose of learning from achieving competence to achieving a reward. At best it distorts learning, making it seem like a chore rather than a delight, at worst it destroys it, even (and perhaps especially) when learners successfully comply with the demands of assessors and get a good grade. Unfortunately, that’s how most educational systems are structured, so the big challenge to all teachers must be to eliminate or at least to massively reduce this deeply pernicious effect. A large number of the pedagogies that we most value are designed to solve problems that are directly caused by credentials. These pedagogies include assessment practices themselves.

With that in mind, before the group’s first meeting I compiled a list of some of the main principles that I adhere to when designing assessments, most of which are designed to reduce or eliminate the structural failings of educational systems. The meeting caused me to reflect a bit more. This is the result:

Principles applying to all assessments

  • The primary purpose of assessment is to help the learner to improve their learning. All assessment should be formative.
  • Assessment without feedback (teacher, peer, machine, self) is judgement, not assessment, pointless.
  • Ideally, feedback should be direct and immediate or, at least, as prompt as possible.
  • Feedback should only ever relate to what has been done, never the doer.
  • No criticism should ever be made without also at least outlining steps that might be taken to improve on it.
  • Grades (with some very rare minor exceptions where the grade is intrinsic to the activity, such as some gaming scenarios or, arguably, objective single-answer quizzes with T/F answers) are not feedback.
  • Assessment should never ever be used to reward or punish particular prior learning behaviours (e.g. use of exams to encourage revision, grades as goals, marks for participation, etc) .
  • Students should be able to choose how, when and on what they are assessed.
  • Where possible, students should participate in the assessment of themselves and others.
  • Assessment should help the teacher to understand the needs, interests, skills, and gaps in knowledge of their students, and should be used to help to improve teaching.
  • Assessment is a way to show learners that we care about their learning.

Specific principles for summative assessments

A secondary (and always secondary) purpose of assessment is to provide evidence for credentials. This is normally described as summative assessment, implying that it assesses a state of accomplishment when learning has ended. That is a completely ridiculous idea. Learning doesn’t end. Human learning is not in any meaningful way like programming a computer or storing stuff in a database. Knowledge and skills are active, ever-transforming, forever actively renewed, reframed, modified, and extended. They are things we do, not things we have.

With that in mind, here are my principles for assessment for credentials (none of which supersede or override any of the above core principles for assessment, which always apply):

  • There should be no assessment task that is not in itself a positive learning activity. Anything else is at best inefficient, at worst punitive/extrinsically rewarding.
  • Assessment for credentials must be fairly applied to all students.
  • Credentials should never be based on comparisons between students (norm-referenced assessment is always, unequivocally, and unredeemably wrong).
  • The criteria for achieving a credential should be clear to the learner and other interested parties (such as employers or other institutions), ideally before it happens, though this should not forestall the achievement and consideration of other valuable outcomes.
  • There is no such thing as failure, only unfinished learning. Credentials should only celebrate success, not punish current inability to succeed.
  • Students should be able to choose when they are ready to be assessed, and should be able to keep trying until they succeed.
  • Credentials should be based on evidence of competence and nothing else.
  • It should be impossible to compromise an assessment by revealing either the assessment or solutions to it.
  • There should be at least two ways to demonstrate competence, ideally more. Students should only have to prove it once (though may do so in many ways and many times, if they wish).
  • More than one person should be involved in judging competence (at least as an option, and/or on a regularly taken sample).
  • Students should have at least some say in how, when, and where they are assessed.
  • Where possible (accepting potential issues with professional accreditation, credit transfer, etc) they should have some say over the competencies that are assessed, in weighting and/or outcome.
  • Grades and marks should be avoided except where mandated elsewhere. Even then, all passes should be treated as an ‘A’ because students should be able to keep trying until they excel.
  • Great success may sometimes be worthy of an award – e.g. a distinction – but such an award should never be treated as a reward.
  • Assessment for credentials should demonstrate the ability to apply learning in an authentic context. There may be many such contexts.
  • Ideally, assessment for credentials should be decoupled from the main teaching process, because of risks of bias, the potential issues of teaching to the test (regardless of individual needs, interests and capabilities) and the dangers to motivation of the assessment crowding out the learning. However, these risks are much lower if all the above principles are taken on board.

I have most likely missed a few important issues, and there is a bit of redundancy in all this, but this is a work in progress. I think it covers the main points.

Further random reflections

There are some overriding principles and implied specifics in all of this. For instance, respect for diversity, accessibility, respect for individuals, and recognition of student control all fall out of or underpin these principles. It implies that we should recognize success, even when it is not the success we expected, so outcome harvesting makes far more sense than measurement of planned outcomes. It implies that failure should only ever be seen as unfinished learning, not as a summative judgment of terminal competence, so appreciative inquiry is far better than negative critique. It implies flexibility in all aspects of the activity. It implies, above and beyond any other purpose, that the focus should always be on learning. If assessment for credentials adversely affects learning then it should be changed at once.

In terms of implementation, while objective quizzes and their cousins can play a useful formative role in helping students to self-assess and to build confidence, machines (whether implemented by computers or rule-following humans) should normally be kept out of credentialling. There’s a place for AI but only when it augments and informs human intelligence, never when it behaves autonomously. Written exams and their ilk should be avoided, unless they conform to or do not conflict with all the above principles: I have found very few examples like this in the real world, though some practical demonstrations of competence in an authentic setting (e.g. lab work and reporting) and some reflective exercises on prior work can be effective.

A portfolio of evidence, including a reflective commentary, is usually going to be the backbone of any fair, humane, effective assessment: something that lets students highlight successes (whether planned or not), that helps them to consolidate what they have learned, and that is flexible enough to demonstrate competence shown in any number of ways. Outputs or observations of authentic activities are going to be important contributors to that. My personal preference in summative assessments is to only use the intended (including student-generated) and/or harvested outcomes for judging success, not for mandated assignments. This gives flexibility, it works for every subject, and it provides unquivocal and precise evidence of success. It’s also often good to talk with students, perhaps formally (e.g. a presentation or oral exam), in order to tease out what they really know and to give instant feedback. It is worth noting that, unlike written exams and their ilk, such methods are actually fun for all concerned, albeit that the pleasure comes from solving problems and overcoming challenges, so it is seldom easy.

Interestingly, there are occasions in traditional academia where these principles are, for the most part, already widely applied. A typical doctoral thesis/dissertation, for example, is often quite close to it (especially in more modern professional forms that put more emphasis on recording the process), as are some student projects. We know that such things are a really good idea, and lead to far richer, more persistent, more fulfilling learning for everyone. We do not do them ubiquitously for reasons of cost and time. It does take a long time to assess something like this well, and it can take more time during the rest of the teaching process thanks to the personalization (real personalization, not the teacher-imposed form popularized by learning analytics aficionados) and extra care that it implies. It is an efficient use of our time, though, because of its active contribution to learning, unlike a great many traditional assessment methods like teacher-set assignments (minimal contribution) and exams (negative contribution).  A lot of the reason for our reticence, though, is the typical university’s schedule and class timetabling, which makes everything pile on at once in an intolerable avalanche of submissions. If we really take autonomy and flexibility on board, it doesn’t have to be that way. If students submit work when it is ready to be submitted, if they are not all working in lock-step, and if it is a work of love rather than compliance, then assessment is often a positively pleasurable task and is naturally staggered. Yes, it probably costs a bit more time in the end (though there are plenty of ways to mitigate that, from peer groups to pedagogical design) but every part of it is dedicated to learning, and the results are much better for everyone.

Some useful further reading

This is a fairly random selection of sources that relate to the principles above in one way or another. I have definitely missed a lot. Sorry for any missing URLs or paywalled articles: you may be able to find downloadable online versions somewhere.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. Retrieved from https://www.jhsph.edu/departments/population-family-and-reproductive-health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-learning.pdf

Boud, D. (2007). Reframing assessment as if learning were important. Retrieved from https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in organizational change and development, 1, 129-169.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3/4), 325-346.

Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3(3), 220-233.

Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes (Kindle ed.). Mariner Books. (this one is worth forking out money for).

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. Retrieved from http://www.alfiekohn.org/blogs/personalized/ (check out Alfie Kohn’s whole site for plentiful other papers and articles – consistently excellent).

Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-Determination research (pp. 183-203). Rochester, NY: The University of Rochester Press.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications. (may be worth paying for if such things interest you).

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Cairo: Ford Foundation. http://www.managingforimpact.org/sites/default/files/resource/outome_harvesting_brief_final_2012-05-2-1.pdf.

BOOK: Teaching Crowds: Learning and Social Media

About the Book

Within the rapidly expanding field of educational technology, learners and educators must confront a seemingly overwhelming selection of tools designed to deliver and facilitate both online and blended learning. Many of these tools assume that learning is configured and delivered in closed contexts, through learning management systems (LMS). However, while traditional “classroom” learning is by no means obsolete, networked learning is in the ascendant. A foundational method in online and blended education, as well as the most common means of informal and self-directed learning, networked learning is rapidly becoming the dominant mode of teaching as well as learning.

In Teaching Crowds, Dron and Anderson introduce a new model for understanding and exploiting the pedagogical potential of Web-based technologies, one that rests on connections — on networks and collectives — rather than on separations. Recognizing that online learning both demands and affords new models of teaching and learning, the authors show how learners can engage with social media platforms to create an unbounded field of emergent connections. These connections empower learners, allowing them to draw from one another’s expertise to formulate and fulfill their own educational goals. In an increasingly networked world, developing such skills will, they argue, better prepare students to become self-directed, lifelong learners.

 

Address of the bookmark: http://www.aupress.ca/index.php/books/120235

Transactional distance and new media literacies

Moore’s theory of transactional distance describes the communications and psychological gulf between learner and teacher in a distance education setting. The theory was formulated in a correspondence era of distance learning and matured in an era where discussion forums and virtual learning environments reduced transactional distance in a closed-group setting that enabled interactions akin to those in a traditional classroom. In recent years the growth of social networking and social interest sites has led to social forms that fit less easily in these traditional formal models of teaching and learning. When the “teacher” is distributed through the network or is an anonymous agent in a set or is an emergent actor formed by collective intelligence, transactional distance becomes a more complex variable. Evolved social literacies are mutated by new social forms and require us to establish new or modified ways of thinking about learning and teaching. In this missive we explore the notion of transactional distance and the kinds of social literacy that are required for or that emerge from network, set, and collective modes of social engagement. We discuss issues such as preferential attachment, confirmation bias, and trust and describe social literacies needed to cope with them.

Address of the bookmark: http://www.mitpressjournals.org/doi/abs/10.1162/IJLM_a_00104#.VEwtAYcfTEI