The limits and limitations of business requirements

Athabasca University’s Digital Governance Committee recently got into a heated debate about whether and why we should support Zoom. It was a classic IT manageability vs user freedom debate and, as is often the way in such things, the suggested resolution was to strike up a working group/sub-committee of stakeholders to identify business requirements that the IT department could use to find an acceptable solution. This approach is eminently sensible, politically expedient, tried-and-tested, and profoundly inadequate.

horse-carAs Henry Ford (probably never) said, “if I’d asked people what they wanted they would have said ‘a better horse'”.

A design approach that starts by gathering business requirements situates the problem in terms of the current solution, which is comprised of layers of solutions to problems caused by other solutions. For simple ‘hygiene’ tech that serves a hard, well-defined business function – leave reporting, accounting, etc – as long as you do properly capture the requirements and don’t gloss over things that matter, that’s normally fine, because you’re just building cogs to make the existing machine work more smoothly. However, for very soft social technologies like meetings, with potentially infinite ways of using them (by which I mean purposes, techniques, ways of assembling them with other technologies, and so on), no list of requirements could even begin to scratch at the surface. The thing about soft technologies – meetings, writing, pencils, pedagogies, programmable computers, chisels, wheels, technologies of fire, groups, poetry, etc – is that they don’t so much solve problems as they create opportunities. They create adjacent possible empty niches. In other words, they are defined by the gaps they leave, much more than the gaps they fill. What happens as a result of them is fundamentally non-deducible. 

Solving different problems, creating different possibles

Meetings are assemblies of vast ranges of technologies and other phenomena, and they serve a vast number of purposes. Meetings are not just one technology but a container for an indefinitely large number of them. They are, though, by and large, solutions to in-person problems, many of which are constrained by physics, physiology, psychology, and other factors that do not apply or that apply differently online. Most webmeeting systems are attempts to replicate the same solutions or (more often) to replicate other webmeeting systems that have already done so, but they are doomed to be pale shadows of the original because there are countless things they cannot replicate, or can only replicate poorly. Among the phenomena that are the default in in-person meetings are, for example:

  • the immense salience brought about by travelling to a location, especially when it involves significant effort (lost in webmeetings);
  • the fact that it forces attention for a sustained period   (most webmeeting software and ways of using it makes inattention much easier);
  • the social bonding that we have evolved to feel in the presence of others (not well catered for in webmeeting software);
  • the focus and meaning that comes from the ‘eventness’ of the occasion (diluted in webmeetings);
  • the ability to directly work together on an issue or artefact (limited in some ways in webmeetings, though potential exists for collaborative construction of digital artefacts);
  • the inability to invisibly escape (easy in most webmeetings);
  • the microexpressions, postures, movements, smells, etc that support communication (largely lost in webmeetings);
  • the social bonding value of sharing food and drink (lost in webmeetings);
  • the blurred boundaries of entering and leaving, the potential to leave together (usually lost in webmeetings);
  • the bonding that occurs in having a shared physical experience, including adversities such as a room that is too hot, roadworks outside, wasps in the room, etc, as well as good things like the smell of good coffee or luxurious chairs (not remotely possible in webmeetings, apart from when the tech fails – but then the meeting fails too);
  • the support for nuances of verbal interaction – knowing when it’s OK to interrupt, being able to sigh, talk at once, etc, not to mention having immediate awareness of who is speaking (webmeetings mostly suck at this);
  • the ability to cluster with others – to sit next to people you know (or don’t know), for instance (rarely an option in most webmeetings, and nothing like as salient or rich in potential as its in-person counterpart even when allowed);
  • the salience of being in a space, with all the values, history, power relationships, and so on that it embodies, from who sits where to which room is chosen (hardly a shadow of this in most webmeetings);
  • the ability to stand up and walk around together (a motion-sickness-inducing experience in webmeetings);
  • the problems and benefits of both over-crowding and excessive sparsity (very different in webmeetings);
  • the means to seamlessly integrate and employ other technologies, including every digital technology as well as paper, dance, desks, chairs, whiteboards, pins, clothing, coffee, doors, etc, etc, etc. (webmeetings offer a tiny fraction of this);
  • and so on.

A few of these might be replicated in current or future webmeeting software, though usually only in caricature. Most simply cannot be replicated at all, even if we could meet as virtual personas in Star Trek’s holodecks. Of course there are also many things that we should be grateful are not replicated in online meetings: conspicuous body odour, badly designed meeting rooms, schedule conflicts, and so on, as well as the unwanted consequences of most of the phenomena above. These, too, are phenomena that the technologies of meetings are designed around.  In-person meetings are incredibly highly-evolved technologies, making use of technological and non-technological phenomena in immensely subtle ways, as well as having layers of counter-technology a kilometre deep, from social mores and manners to Roberts’ rules, from meeting tables to pens and note-taking strategies. Much of the time we don’t even notice that there are any technologies involved at all (as Danny Hillis quipped, ‘technology’ is anything invented after you were born).

Webmeetings, though, also have distinctive phenomena that can be exploited, such as:

  • the ease of entering and leaving (so breaks are easier to take, they don’t need to last a long time, people can dip in and out, etc);
  • the automation of scheduling and note-taking;
  • the means to record all that occurs;
  • the means to directly share digital tools;
  • the fact that people occupy different spaces (often with tools at their disposal that would be unavailable in a shared meeting space);
  • the captions for the hard of hearing;
  • the integrated backchannels of text chat.

These are different kinds of problem space with different adjacent possibles as well as different constraints. It therefore makes no sense to blindly attempt to replicate in-person meetings when the problems and opportunities are so different. We don’t (or shouldn’t) teach online in the same way we teach in the classroom, so why should we try to use meetings in the same way? For that matter, why have meetings at all?

Dealing with the hard stuff

Some constraints are quite easy to specify. If a matter under discussion needs to be kept private, say, that limits the range of options, albeit that, for such a soft technology as a meeting, privacy needs may vary considerably, and what works for one context may fail abysmally for another. Similarly for security, accessibility, learnability, compatibility, interoperability, cost, reliability, maintainability, longevity, and other basic hygiene concerns. There are normally hard constraints defining a baseline, but it is a fuzzy baseline that can be moved in different contexts for different people and different uses. No one wants unreliable, insecure, expensive, incompatible, unusable, buggy, privacy abusing software but most of us nonetheless use Microsoft products.

It is also not completely unreasonable to look for specific known business requirements that need to be met. However, there are enormous risks of duplicating solutions to non-existent problems. It is essential, therefore, to try to find ways of understanding the problems themselves, as much as possible in isolation from existing solutions. It would be a bad requirement to simply specify that people should be able to see and hear one another in real-time, for example: that is a technological solution based on the phenomena that in-person meetings use, not a requirement. It is certainly a very useful phenomenon that might be exploited in any number of ways (we know that because our ancestors have done it since before humans walked the planet) but it tells us little about why the phenomenon matters, or what it is about it that matters.

It would be better, perhaps, to ask people what is wrong with in-person meetings. It still situates the requirements in the current problem space, but it looks more closely at the source rather than the copy. It makes it easier to ask what purposes being able to see and hear one another during in-person meetings serve, what phenomena it provides, on what phenomena (including those provided by other technologies) it depends, and what depends on it. From that we may uncover the business requirements that seeing and hearing other people actually meet. However, it is incredibly tricky to ask such questions in the abstract: the problem space is vast, complex, diverse, and deeply bound up in what we are familiar with, not what is possible.

It might help to make the familiar unfamiliar, for instance, by holding in-person meetings wearing blindfolds, or silently, or to attempt to conduct a meeting using only sticky notes (approaches I have used in my own teaching about communication technologies, as it happens). This kind of exercise forcibly creates a new problem space so that people can wonder about what is lost, what is gained, reasons for doing things, and so on. If you do enough of that, you might start to uncover what matters, and (perhaps) some of the reasons we have meetings in the first place.

Exploring the adjacent possible

Perhaps most importantly, though, soft technologies are not just solutions to problems. Soft technologies are, first and foremost, creators of opportunities, the vast majority of which we will never begin to imagine. Soft technology design is therefore, and must be, a partnership between the person and the technology: it’s not just about creating a tool for a task but about having a conversation with that tool, asking what it can do for us and wondering where it might lead us. What’s interesting about the ubiquitous backchannel feature of webmeetings, for instance, is that it did not find its way into the software as a result of a needs assessment or analysis of business requirements. It was, instead, an early (and deeply imperfect) attempt at replicating what could be replicated of synchronous meetings before multimedia communication became possible. When designing early web conferencing systems, no one said ‘we need a way of typing so that others can see it’. They looked at what could be done and said ‘hey, we can use that’. The functionality persisted and has become nearly ubiquitous because it’s easy to implement and obviously useful. It’s an exaptation, though, not the product of a pre-planned intentional design process. It’s a side-effect of something else we did – a poor solution to an existing problem – that created new phenomena we could co-opt for other purposes. New adjacent possible empty niches emerged from it.

One way to explore such niches would be to give people the chance to play with a wide range of existing ways of addressing the same problem space. A lot of people have turned their attention to these issues, so it makes sense to mine the creativity of the crowd. There are systems like Discord or MatterMost, that represent a different category of hybrid asynchronous/synchronous tool, for instance, blurring the temporal boundaries. There are spatial metaphor systems with isometric interfaces like Spatial, or Ovice, which can allow more intuitive clustering, perhaps contributing to a greater sense of the presence of others, while enabling novel approaches to (say) voting, and so on. There are immersive systems that more literally replicate spaces, like Mozilla Hubs or OpenSim. I hold out little hope for those, but they do have some non-literal features – especially in ways they allow impossible spaces to be created – that are quite interesting. There are instant messengers like Telegram or Signal, that offer ambient awareness as well as conventional meeting support (MS Teams, reflecting its Skype origins, has that too). There are games and game-like environments like Gather or Minecraft, that create new kinds of world as well as providing real-time conferencing features. And there are much smarter webmeeting systems like Around (that largely solves almost all audio problems, that – crucially – can make the meeting a part of a user’s environment rather than a separate space for gathering, that rethinks text chat as a transient, person-focused act rather than a separate text-stream, that makes working together on a digital artefact a richly engaging process, that automatically sends a record to participants, and more).  And there’s a wealth of research-based systems that we have built over the past few decades, including many of my own, that do things differently, or that use different metaphors. Computer-supported collaborative argumentation tools, for instance, or systems that leverage social navigation (I particularly love Viégas’s and Donath’s ChatCircles from the late 1990s, for instance), and so on. They all make new problems, and all have flaws of one kind or another, but thinking about how and why they are different helps to focus on what we are trying to do in the first place.

Perhaps the best of all ways to explore those adjacent possible empty niches is to make them: not to engineer it according to a specification, but to tinker and play. I’ve written about this before (e.g. here and, paywalled, here, summarized by Stefanie Panke here). Tinkering as a research methodology is a process of exploration not of what exists but of what does not. It’s a journey into the adjacent possible, with each new creation or modification creating new adjacent possibles, a step by step means of reaching into and mapping the unknown. We don’t all have the capacity (in skills, time, or patience) to create software from scratch, but we can assemble what we already have. We can, for instance, try to add plugins to existing systems: it is seldom necessary to write your own WordPress plugin, for example, because tens of thousands of people have already done so. Or we can make use of frameworks to construct new systems: the Elgg system underpinning the Landing, for example, does require some expertise to build new components, but a lot can be achieved by assembling and/or modifying what others have built. Or, if standards are followed, we can assemble services as needed: there are standards like xcon, XMPP, Jabber, IRC, and so on that make this possible. And we don’t need to create software or hardware at all in order to dream. Hand-drawn mockups can create new possibilities to explore. Small steps into the unknown are better than no steps at all.

Stop looking for solutions

Webmeetings that attempt to replicate their in-person inspirations are unlikely to ever afford the flexibility of in-person meetings, because they have fewer phenomena to orchestrate and we are never going to be as adept at using them. The gaps they leave for us to fill are smaller, and our capacity to fill those gaps is less well-developed. However, digital systems can provide a great many new and different phenomena that, with creativity and inspiration, may meet our needs much better. Without the constraints of physical spaces we can invent a new physics of the digital. As long as we treat the problem as one of replicating meetings then it makes little difference what we choose: Zoom, Teams, Webex, Connect, BBB, Jitsi, whatever – the feature set may vary, there may be differences in reliability, security, cost, etc but any of them will do the job. The problem is that it is the wrong job. We already pay for and use at least three major systems for synchronous meetings at AU, as well as a bunch of minor ones, and that is nothing like enough. Those that begin to depart from the replication model – Around being my current favourite – are a step in the right direction, while those that double down on it (notably most immersive environments) are probably a step in the wrong direction. It is not about going forward or backward, though: it is about going sideways.

It is not too tricky to experiment in this particular field. For most digital systems we create our decisions normally haunt us for years or decades, because we become locked in to them with our data. Synchronous technologies can, with provisos, be swapped around and changed at will. Sure, there can be issues with recording and transcripts, there can be a training burden, contracts can be expensive and hard to escape, and tech support may be a little more costly but, for the most part, if we don’t like something then we can drop it and try something else. 

I don’t have a solution to choosing or making the right piece of software for AU’s needs, because there isn’t one. There are countless possible solutions, none of which will suit everyone, many of which will provide parts that might be useful to most people, and all of which will have parts or aspects that won’t. But I do know that the way to approach the problem is not to have meetings to determine business requirements. The solution is to find ways of discovering the adjacent possible, to seek inspiration, to look sideways and forwards instead of backwards. We don’t need simple problem-solving for this kind of situation (or rather, it is quite inadequate on its own): we need to find ways to dream, ways to wonder, ways to engage in the act of creation, ways to play.

 

A few thoughts on learning management systems, and on integrated learning environments and their implementation

Why do we build digital learning systems to mimic classrooms?

It is understandable that, when we teach in person, we have to occupy and make different uses of the same or similar environments like classrooms, labs, workshops, lecture theatres, and offices. There are huge financial, physical, and organizational constraints on making the environment fit the task, so it would be madness to build a whole new classroom every time we wished to run a different class.

Online, we could build anything we like

But why do we do the same when we teach online? There are countless tools available and, if none are suitable, it is not too hard to build them or modify them to suit our needs. Once they are built, moving between them just takes a tap of a screen or the click of a mouse. Heck, you can even occupy several of them at once if you have a decent monitor or more than one device.

So why don’t we do this?

Here are a few of the more obvious reasons that using the perfect app for the context of study rarely happens:

  • Teachers’ lack of knowledge of the options (it takes time and effort to discover what’s available).
  • Teachers’ lack of skill in using them (most interesting tools have a learning curve, and that gets steeper in inverse proportion to the softness and diversity of the toolset, so most teachers don’t even know how to make the most of what they already have).
  • Lack of time and/or money for development (a real-life application is what it contains, not just the shell that contains it, and it is not always as easy to take existing stuff and put it in a new tool as it might be in a physical space).
  • Costs and difficulties in management (each tool adds costs in managing faults, configuration, accounting for use, performance, and security).
  • Cognitive load involved for learners in adapting to the metaphors, signposts, and methods needed to use the tool itself.

All of these are a direct consequence of the very diversity that would make us want to use different apps in the first place. This is a classic Faustian bargain in which the technology does what we want, and in the process creates new problems to solve.  Every virtual system invents at least some of the dynamics of how people and things interact with it and within it. In effect, every app has its own physics. That makes them harder to find out about, harder to learn, harder to develop, costlier to manage, and more difficult to navigate than the static, fixed facilities found in particular physical locations. They are all different, there are few if any universals, and any universal today may become a conditional tomorrow. Gravity doesn’t necessarily work the same way in virtual systems.

image of a pile of containersAnd so we get learning management systems

The learning management system (LMS) kind of deals with all of these problems: poorly, harmfully, boringly, and painfully, but it does deal with them. Currently, most of the teaching at Athabasca University is through the open source Moodle LMS, lightly modified by us because our needs are not quite like others (self-pacing and all that). But Moodle is not special: in terms of what it does and how it does it, it is not significantly different from any other mainstream LMS – Blackboard, Brightspace, Canvas, Sakai, whatever.

Almost every LMS essentially automates the functions, though not exactly the form, of traditional classrooms. In other parts of the world people prefer to use the term ‘managed learning environment’ (MLE) for such things, and it is the most dominant representative of a larger category of systems usually described as virtual learning environments (VLEs) that also includes things like MOOs (multi-user dungeons, object oriented), immersive learning environments, and simpler web-based teaching systems that replicate aspects of classrooms such as Google Classroom or Microsoft’s gnarly bundle of hastily repurposed rubbish for teaching that I’m not sure even has a name yet. Notice the spatial metaphors in many of these names.

Little boxes made of ticky tacky

The people who originally designed LMSs back in the 90s (I did so myself) based their designs on the functions and entities found in a traditional university because that was their context, and that was where they had to fit. Metaphorically, an LMS or MLE is a big university building with rather uniform classrooms, with perhaps a yard where you can camp out with a few other systems (plugins, LTI hooks, etc) that conform to its requirements and that are allowed in to classrooms when invited, and a few doors and gateways (mainly hyperlinks) linking it circuitously or in jury-rigged fashion to other similarly weakly connected buildings (e.g. places to register, places to seek support, places to talk to an advisor, places to complain, places to find books, and so on). It doesn’t have metaphorical corridors, halls, common rooms, canteens, yards, libraries or any of the other things that normally make up a physical university. You rarely get to even be aware of other classrooms beyond those you are in. Some people (me in a past life) might give classrooms cute names like ‘the learning cafe’ but it’s still just another classroom. You teleport from one classroom to the next because what happens in corridors (really a big lot of incredibly important pedagogically useful stuff, as it happens) is not perceived by the designers as a useful classroom function to be automated or perhaps, more charitably, they just couldn’t figure out how to automate that.

Reified roles

It’s a very controlled environment where everyone has a programmatically enforced role (mostly reflecting traditional educational roles), that may vary according to the room, but that are far less fluid than those in physical spaces. There are strong hierarchies, and limited opportunities for moving between them. Some of those hierarchies are new: the system administrator, for instance, has way more power than anyone in a physical university to determine how learning happens, like an architect with the power to move walls, change the decor, add extensions, and so on, at will. The programmers of the system are almost god-like in their command of its physics. But the ways that they give teachers (or learning designers, or administrators) control, as designers, directors, and regulators of the classroom, are perhaps the most pernicious. In a classroom a teacher may lead (and, by default, usually does). In an LMS, a teacher (or someone playing that role) must lead. The teacher sees things that students cannot, and controls things that the students may not. A teacher configures the space, and determines with some precision how it will be used. With a lot of effort and risk, it can be made to behave differently, but it almost never is.

Functions are everything

An LMS is typically built along functional lines, and those functions are mostly based on loose, superficial observations of what teachers and students seem to do in physical classrooms. The metaphorical classrooms are weird, because they are structured by teaching (seldom learning) function rather than along pedagogical lines: for instance, if you want to talk with someone, you normally need to go to a separate enclosed area inside the classroom or leave a note on the teacher’s desk. Same if you want to take a test, or share your work with others. Another function, another space. Some have many little rooms for different things. Lectures are either literally that (video recordings) or (more usefully, from a learning perspective), text and images to be read on screen, based on the assumption that the only function of lectures is information transmission (it is so very, very much not – that’s its least useful and least effective role). There’s seldom a chance to put even put up your hand to question something. Notices can usually only be pinned on the wall by teachers. Classroom timetables are embodied in software because of course you need a rigid and unforgiving timetable in a medium that sells itself on enabling learning anywhere, any time. Some, including Moodle, will allow you to break up the content differently, but it’s still another timetable; just a timetable without dates. It’s still the teacher who sets the order, pacing and content.

Robot overlords

It’s a high-tech classroom. There are often robots there that are programmed to make you behave in ways determined by those higher in the hierarchy (sometimes teachers, sometimes administrators, sometimes the programmers of the software). For instance, they might act as gatekeepers that prevent you from moving on to the next section before completing the current one, or they might prevent you submitting work before or after a specified date. They might mark your work. There are surveillance cameras everywhere, recording your every move, often only accessible to those with more powerful roles (though sometimes a robot or two might give you a filtered view of it).

Beginnings and ends

You can’t usually go back and visit when your course is over because someone decided it would be a good idea to set opening and closing enrolment dates and assumed that, when they were done, the learning was done (which of course it never is – it keeps on evolving long after explicit teaching and testing occurred). Again, it’s because physical classes are scheduled and terms come to an end because they must be, not because it makes pedagogical sense. And, like almost everything, you can override this default, but hardly anyone ever does, because it brings back those Faustian bargains, especially in manageability.

Dull caricatures of physical spaces

Basically, the LMS is an automated set of metaphorical classrooms that hardens many of the undesirable by-products of educational systems in software in brain-dead ways that have little to do with how best to teach, and that stretch the spatial metaphors that inform it beyond breaking point. Each bit of automation and each navigational decision hardens pedagogical choices. For all the cozy metaphors, programmers invent rather than replicate physics, in the process warping reality in ways that do no good and much harm. Classrooms solved problems of physics for in-person teaching and form part of a much larger structure that has evolved to teach reasonably well (including corridors, common rooms, canteens, and libraries, as it happens). Their more visible functions are only a part of that and, arguably, not the main part. There is much pedagogy embedded in the ways that physical universities, whether by accident or design, have evolved over centuries to support learning in every quadrangle and nook of a coffee shop. LMSs just focus on a limited subset of teaching roles, and empower the teacher in ways that caricature their already excessive dominance in the classroom (which only occurred because it had to, thanks to physics and the constraints it imposed).

LMSs are crap, but they contain recognizable semblances of their physical counterparts and just enough configurability and flexibility to more or less work as teaching tools, a bit, for everyone, almost no matter what their level of digital proficiency might be. They more or less solve the Faustian bargains listed earlier, but they do so by stifling what we wanted and should have been able to do in the first place with online tools, in the process creating new and quite horrific problems, as well as demolishing most of what makes physical universities work in the first place. It never has been true that virtual learning environments are learning environments – they are only ever parts of them – and there are places to escape from them, such as the Landing, other virtual systems, or even just plain old email, but then all those Faustian bargains come back to haunt us again. There has to be a better way.

Beyond the LMS

Cognisant of the issues, Athabasca University is now some way down the path to developing its own distinctive solutions to these problems, in a multi-year multi-million-dollar initiative known as (following the spatial metaphor) the Integrated Learning Environment (ILE). The ILE is not an application. It is an umbrella term for a lot of different, usually independent systems working together as one. Though some of the most interesting opportunities are still only loosely imagined, perhaps because they cause problems that are fiendishly hard to solve (e.g. how can we integrate systems that we build ourselves without creating risks for the rest of the ILE, and what happens when they need to be maintained?) a lot of progress is being made on the non-teaching foundations on which the rest depends (student admin systems, support tools, procedures, etc), as well as on the most visible and perhaps the biggest of its parts, BrightSpace, a proprietary commercial LMS that is meant to replace Moodle, for no obvious pedagogical or technical reasons (it’s no better). It might make economic sense. I don’t know, but I do know that open source software typically costs a fair bit to own, albeit because of the things that make it a much better idea (freedom, flexibility, ownership, etc). There is probably a fair bit of time and money being spent with Desire2Learn (makers of Brightspace) on the things that we spent a fair bit of time and money on many years ago to make Moodle a bit less classroom-like. The choice no doubt has something to do with how reliably and easily it can be made to work with some of the other proprietary commercial systems that someone has decided will make up the ILE. It bothers me greatly that we are not trying hard to choose open source solutions, for reasons that will become clearer in the rest of this post. However, (pedagogically speaking) all the mainstream LMSs are much of a muchness, making the same mistakes as one another in very similar ways, so it probably won’t wreck too much of what we already do within Moodle. But, on its own, it won’t move us much further forward and we could do it better. That’s what the ILE is supposed to do – to make the LMS just a part of a much larger teaching environment, intimately connected with the rest of what the university does for or with students, and extensible with new and better ways of learning, teaching, and assessing learning.

picture of lego bricksLego bricks make poor metaphors

When we were first imagining the ILE, though the approach was admirably participative, engaging much of the university community, I was very worried by the things we were encouraged to focus on. It was all about the functionality, the usability, the design, the tools, the pedagogies, the business systems that supported them. Those things matter, for sure, and should be not be ignored, but they should and will change and grow all the time: in fact, part of the point of building this thing is to do just that. Using the city metaphor, pretty much all that we (collectively) considered were the spaces (the rooms, mainly), and the stuff that goes on inside them, much like LMS designers thought of universities as just collections of classrooms in which teaching functions were performed. Space and stuff are, not uncoincidentally, exactly what Stewart Brand identified long ago as inevitably being the fastest-changing, most volatile parts of any town or city (after site, structure, skin, and services). I’ve written a fair bit on the universality of this principle across all systems. It’s a solid structural principle that applies as much to ecosystems and educational systems as to cities. As Brand observes himself, drawing from O’Neill et al (1986), the larger, slower-changing elements of any system affect the smaller, faster-changing more than vice versa. This is for much the same reasons that path dependencies set in. It’s about the prior providing the context for what follows. Flexible things have to fit into the gaps left by less flexible, older, pre-existing things. In physical spaces, of course these tend to be bigger and/or slower, but the same is true in virtual spaces, where size seldom matters that much, but hardness (inflexibility, brittleness) really does. Though lip service was paid to the word ‘integrated’ in our discussions,  I had the strong feeling that the kind of integration we had in mind was that of a Lego set. In fact, I think we were aiming to find a ‘Lego Athabasca University’ set, with assembly instructions and a picture on the box. The vendors who came to talk with us made much of how effectively they could do that, rather than how effectively they could make it possible for others to do that.

Metaphors matter. Lego bricks have to fit together tightly, in pre-specified ways, especially if you are following a plan. If you want to move them around, you have to dismantle a bit of the structure to fit them in. It’s difficult to integrate things that are not bricks, or that are made by different toy companies to work in different ways. At best you get what Brand calls ‘magazine architecture’, or ‘no road’ architecture, beautiful, fit for purpose, intricate and solid, but slow to learn. Lego is not a terrible way to build, compared with buying everything pre-assembled, but it could be improved.

Signals and boundaries

Drawing inspiration from John Holland’s brilliant last work, Signals & Boundaries, I tried to make the case that, instead, we should be focusing on the boundaries (the interfaces between the buildings and the rest of the city), and the signals that pass between them (the people, the messages, etc, the forms they take and how they move around). In Brand’s terms, I wanted us to be thinking about skin and services, and perhaps even structure, though site – Athabasca University – was a given. Though a few people nodded in agreement, I think it mainly fell on deaf ears. We wanted oven-ready solutions, not the infrastructure to enable those solutions. Though the city metaphor works well, because we are talking about human constructions, others would result in similar ways of thinking: cells in bodies, organisms in ecosystems, brains, termite mounds, and so on. All are organized by boundaries (at many levels of hierarchy) and the signals that pass between them.

The Lego set metaphor – whether deliberately or not – seems to have prevailed for now. A lot of old buildings are being slated for demolition and a lot of new virtual buildings are now being erected as part of this development, many of them chosen not because of problems with existing buildings but so that they can more easily connect together and live in the same cloud. This will very likely work, for now, but it is not cheap and it is not flexible, especially given the fact that most of it is not open so, like a rental property, we are not allowed to fix things, add utilities, change the walls, etc, and we are wholly dependent on the landlords being nice to us and each other (knowing that some – ahem, Microsoft – have a long history of abusing their tenants). Those buildings will age. We will find them cramped. Some will age faster than others, and will have to be modified to keep up, perhaps at high cost. Companies renting them might go out of business or change their terms so we might have to demolish the buildings and rent/make new ones. We will be annoyed at how they do things, usually without asking us. We will hate the landlords who dictate what we can do and how we can do it, and who will keep upping the rent while not doing what we ask. We will want more, and the only way to get it will be to build extensions, buy new brick sets, if it is not enough to pay someone to remodel the interiors (and it won’t be). Of course, because most of the big structural elements will not be open source, we will not be able to do that ourselves.

What the ILE really should be

The ILE is, I think, poorly named, because it should not be an environment at all. Following the building metaphor, the ILE is (or should be) more like the system that connects a lot of buildings, bringing them together into a coherent, safe, livable community. It’s infrastructure and services; it is the roads, the traffic signals, the doors, the sidewalks, the water pipes, the waste pipes, the electricity, the network cables; it is the services – fire, police, schools, traffic control, etc; it is all the many rules, standards, norms and regulations that make them work together to help make an environment in which people can live, work, play, and grow. It’s part of the environment – the part that makes it work – but it is not the environment itself. The environment itself is Athabasca University, not just the tools, processes, and systems that support its functions. That includes, most importantly, the people who are part of the university, or who are visitors to it, who are not just users of the environment or dwellers in its walls, but who are or should be the most significant and visible parts of it, just as trees are part of the environment of forests, not users of the forest. Those people live in physical as well as other virtual environments (social media, Word documents, websites, etc) that the ILE can connect together too, to make them a part of it, so the spatial metaphor gets weird at this point. The ILE makes environmental boundaries fuzzy, permeable, and shifting. It’s not an ILE, it’s an ILI – an integrated learning infrastructure.

If we focused on the connections and interfaces, and on how information and processes need to pass across them, and if we thought hard about the nature of those signals, then we could build a system that is resilient, that adapts, that lasts, that grows, that evolves, with parts that we can seamless replace or improve because the interfaces – the building facades, the mains pipes, the junction boxes, etc – will mostly stay the same, evolving slowly as they should. This is about strategy, not planning,  a way of thinking about systems rather than a sequence of things to do.

Some of the key people involved in the process realize this. They are talking about standards, protocols, and projects to build interfaces between systems, and imagining future needs, though they are inevitably distracted by the process of renting Lego bricks, so I am not sure how much they will be able to stay focused on that. I hope they prevail over those who think they are building a set of classrooms and tightly connected admin offices out of self-contained interlocking bricks because our future depends on getting it right. We are aiming to grow. It just takes one critical piece in the Lego building to fail to support that, and the rest falls apart like a… well, like a pile of bricks.

References

Brand, S. (1997). How buildings learn. Phoenix Illustrated. https://www.penguinrandomhouse.ca/books/320919/how-buildings-learn-by-stewart-brand/9780140139969

Holland, J. H. (2012). Signals and Boundaries: Building Blocks for Complex Adaptive Systems. MIT Press.  https://mitpress.mit.edu/books/signals-and-boundaries

O’Neill, R.V., DeAngelis, D.L, Waide, J. B., & Allen, T. F. H. (1986). A Hierarchical Concept of Ecosystems. Princeton University Press. http://www.gbv.de/dms/bs/toc/025157787.pdf

Postman, N. (1998). Five things we need to know about technological change. Denver, Colorado, 28.  https://student.cs.uwaterloo.ca/~cs492/papers/neil-postman–five-things.html

Educational technology: what it is and how it works

https://rdcu.be/ch1tl

This is a link to my latest paper in the journal AI & Society. You can read it in a web browser from there, but it is not directly downloadable. A preprint of the submitted version (some small differences and uncorrected errors here and there, notably in citations) can be downloaded from https://auspace.athabascau.ca/handle/2149/3653. The published version should be downloadable for free by Researchgate members.

This is a long paper (about 10,000 words), that summarizes some of the central elements of the theoretical model of learning, teaching and technology developed in my recently submitted book (still awaiting review) and that gives a few examples of its application. For instance, it explains:

  • why, on average researchers find no significant difference between learning with and without tech.
  • why learning styles theories are a) inherently unprovable, b) not important even if they were, and c) a really bad idea in any case.
  • why bad teaching sometimes works (and, conversely, why good teaching sometimes fails)
  • why replication studies cannot be done for most educational interventions (and, for the small subset that are susceptible to reductive study, all you can prove is that your technology works as intended, not whether it does anything useful).

Abstract

This theoretical paper elucidates the nature of educational technology and, in the process, sheds light on a number of phenomena in educational systems, from the no-significant-difference phenomenon to the singular lack of replication in studies of educational technologies.  Its central thesis is that we are not just users of technologies but coparticipants in them. Our participant roles may range from pressing power switches to designing digital learning systems to performing calculations in our heads. Some technologies may demand our participation only in order to enact fixed, predesigned orchestrations correctly. Other technologies leave gaps that we can or must fill with novel orchestrations, that we may perform more or less well. Most are a mix of the two, and the mix varies according to context, participant, and use. This participative orchestration is highly distributed: in educational systems, coparticipants include the learner, the teacher, and many others, from textbook authors to LMS programmers, as well as the tools and methods they use and create.  From this perspective,  all learners and teachers are educational technologists. The technologies of education are seen to be deeply, fundamentally, and irreducibly human, complex, situated and social in their constitution, their form, and their purpose, and as ungeneralizable in their effects as the choice of paintbrush is to the production of great art.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/8692242/my-latest-paper-educational-technology-what-it-is-and-how-it-works

Are experienced online teachers best-placed to help in-person teachers cope with suddenly having to teach online? Maybe not.

lecturingI recently downloaded What Teacher Educators Should Have Learned From 2020. This is an open edited book, freely downloadable from the AACE site, for teachers of teachers whose lives were disrupted by the sudden move to emergency remote teaching over the past year or so.  I’ve only skimmed the contents and read a couple of the chapters, but my first impressions are positive. Edited by Richard Ferdig and Kristine Pytash, It springs from the very active and engaged AACE SITE community, which is a good indicator of expertise and experience. It seems well organized into three main sections:

  1.         Social and Emotional Learning for Teacher Education.
  2.         Online Teaching and Learning for Teacher Education.
  3.         eXtended Reality (XR) for Teacher Education

I like the up-front emphasis on social and emotional aspects, addressing things like belongingness, compassion, and community, mainly from theoretical/model-oriented perspectives, and the other sections seem wisely chosen to meet practitioner needs. The chapters adopt a standardized structure:

  • Introduction. 
  • What We Know. 
  • Lessons Learned for Research. 
  • Lessons Learned for Practice. 
  • What You Should Read. 
  • References

Again, this seems pretty sensible, maintaining a good focus on actionable knowledge and practical steps to be taken. It’s not quite a textbook, but it’s a useful teach-yourself resource with good coverage. I look forward to dipping into it a bit more deeply. I expect to find some good ideas, good practices, and good theoretical models to support my teaching and my understanding of the issues. And I’m really pleased that it is being released as an open publication: well done, AACE, for making this openly available.

But I do wonder a little about who else will read this.

Comfort zones and uncomfortable zones

The other day I was chatting with a neighbour who teaches a traditional hard science subject at one of the local universities, who was venting about the problems of teaching via Zoom. He knew that I had a bit of interest and experience in this area, so he asked whether I had any advice. I started to suggest some ways of rethinking it as a pedagogical opportunity, but he was not impressed. Even something as low-threshold and straightforward as flipping the classroom or focusing on what students do rather than what he has to tell them was a step too far. He patiently explained that he has classes with hundreds of students and fixed topics that they need to learn, and he really didn’t see it as desirable or even possible to depart from his well-tried lecture format. At least it would be too much work and he didn’t have the time for it. I did try to push back on that a bit and I may have mentioned the overwhelming body of research that suggests this might not be a wise move, but he was pretty clear and firm about this.  What he actually wanted was for someone to make (or tell him how to make) the digital technology as easy and as comfortably familiar as the lecture theatre, and that would somehow make the students as engaged as he perceived them to normally be in his lectures, without notably changing how he taught. The problem was the darn technology, not the teaching. I bit my tongue at this point. I eventually came up with a platitude or two about trying to find different ways to make learning visible, about explicitly showing that he cares, about taking time to listen, about modelling the behaviour he wanted to see, about using the chat to good advantage, and about how motivation differs online and off, but I don’t think it helped. I suspect that the only things that really resonated with him were suggestions about how to get the most out of a webcam and a recommendation to get a better microphone.

Within the context in which he usually teaches, he is probably a very good teacher. He’s a likeable person who clearly cares a lot about his students, he knows a lot about his subject, and he knows how to make it appealing within the situation that he normally works. His courses, as he described them, are very conventional, relying a lot on the structure given to them by the industry-driven curriculum and the university’s processes, norms, and structures, and he fills his role in all that admirably. I think he is pretty typical of the vast majority of teachers. They’re good at what they do, comfortable with how they do it, and they just want the technology to accommodate them continuing to do so without unnecessary obstacles.

Unfortunately, technology doesn’t work that way.

The main reason it doesn’t work is very simple: technologies (including pedagogies) affect one another in complex and recursive ways, so (with some trivial exceptions) you can’t change one element (especially a large element) and expect the rest to work as they did before.  It’s simple, intuitive, and obvious but unless you are already well immersed in both systems theories and educational theory, really taking it to heart and understanding how it must affect your practice demands a pretty big shift in weltanschauung, which is not the kind of thing I was keen to start while on my way to the store in the midst of a busy day.

To make matters worse, even if teachers do acknowledge the need to change, their assumption that things will eventually (maybe soon) return to normal means that they are – reasonably enough –  not willing and probably not able to invest a lot of time into it. A big part of the reason for this is that, thanks to the aforementioned interdependencies, they are probably running round like blue-arsed flies just trying to keep things together, and filling their time with fixing the things that inevitably break in the process. Systems thrive on this kind of self-healing feedback loop. I guess teachers figure that, if they can work out how to tread water until the pandemic has run its course, it will be OK in the end.

If only.

Why in-person education works

The hallmark technologies (mandatory lectures, assignments, grades, exams, etc, etc) of in-person teaching are worse than awful but, just as a talented musician can make beautiful noises with limited technical knowledge and sub-standard instruments, so there are countless teachers who use atrocious methods in dreadful contexts but who successfully lead their students to learn. As long as the technologies are soft and flexible enough to allow them to paper over the cracks of bad tools and methods with good technique, talent, and passion, it works well enough for enough people enough of the time and can (with enough talent and passion) even be inspiring.

It would not work at all, though, without the massive machinery that surrounds it.

An institution (including its systems, structures, and tools) is itself designed to teach, no matter how bad the teachers are within it. The opportunities for students to learn from and with others around them, including other students, professors, support staff, administrators, and so on; the supporting technologies, including rules, physical spaces, structures, furnishings, and tools; the common rooms, the hallways, the smokers’ areas (best classrooms ever), the lecture theatres, the bars and the coffee shops; the timetables that make students physically travel to a location together (and thus massively increase salience); the notices on the walls; the clubs and societies; the librarians, the libraries, the students reading and writing within those libraries, echoing and amplifying the culture of learning that pervades them; the student dorms and shared kitchens where even more learning happens; the parties; even the awful extrinsic motivation of grades, teacher power, and norms and rules of behaviour that emerged in the first place due to the profound motivational shortcomings of in-person teaching. All of this and more conspires to support a basic level of at least mediocre (but good enough) learning, whether or not teachers teach well. It’s a massively distributed technology enacted by many coparticipants, of which designated teachers are just a part, and in which students are the lead actors among a cast of thousands. Online, those thousands are often largely invisible. At best, their presence tends to be highly filtered, channeled, or muted.

Why in-person methods don’t transfer well online

When most of that massive complex machinery is suddenly removed, leaving nothing but a generic interface better suited to remote business meetings than learning or, much worse, some awful approximation of all the evil, hard, disempowering technologies of traditional teaching wrapped around Zoom, or nightmarishly inhuman online proctoring systems, much of the teaching (in the broadest sense) disappears with it. Teaching in an institution is not just what teachers do. It’s the work of a community; of all the structures the community creates and uses; of the written and unwritten rules; of the tacit knowledge imparted by engagement in a space made for learning; of the massive preparation of schooling and the intricate loops that connect it with the rest of society; of attitudes and cultures that are shaped and reinforced by all the rest.  It’s no wonder that teachers attempting to transfer small (but the most visible) parts of that technology online struggle with it. They need to fill the ever-widening gaps left when most of the comfortable support structures of in-person institutions that made it possible in the first place are either gone or mutated into something lean and hungry. It can be done, but it is really hard work.

More abstractly, a big part of the problem with this transfer-what-used-to-work-in-person approach is that it is a technology-first approach to the problem that focuses on one technology rather than the whole. The technology of choice in this case happens to be a set of pedagogical methods, but it is no different in principle than picking a digital tool and letting that decide how you will teach. Neither makes much sense. All the technologies in the assembly – including pedagogies, digital tools, regulations, designs, and structures – have to work together. No single technology has precedence, beyond the one that results from assembling the rest. To make matters worse, what-used-to-work-in-person pedagogies were situated solutions to the problems of teaching in physical classrooms, not universally applicable methods of teaching. Though there are some similarities here and there, the problems of teaching online are not at all the same as those of in-person teaching so of course the solutions are different. Simply transferring in-person pedagogies to an online context is much like using the paddles from a kayak to power a bicycle. You might move, but you won’t move far, you won’t move fast, you won’t move where you want to go, and it is quite likely to end in injury to yourself or others.

Such problems have, to a large extent, been adequately solved by teachers and institutions that work primarily online. Online institutions and organizations have infrastructure, processes, rules, tools, cultures, and norms that have evolved to work together, starting with the baseline assumption that little or none of the physical stuff will ever be available. Anything that didn’t work never made it to first base, or has not survived. Those that have been around a while might not be perfect, but they have ironed out most of the kinks and filled in most of the gaps. Most of my work, and that of my smarter peers, begins in this different context. In fact, in my case, it mainly involves savagely critiquing that context and figuring out ways to improve it, so it is yet another step removed from where in-person teachers are now.

OK, maybe I could offer a little advice or, at least, a metaphor

Roughly 20 years ago I did share a similar context. Working in an in-person university, I had to lead a team of novice online teachers from geographically dispersed colleges to create and teach a blended program with 28 new online courses. We built the whole thing in 6 months from start to finish, including the formal evaluations and approvals process. I could share some generic lessons from what I discovered then, the main one being to put most of the effort into learning to teach online, not into designing course materials. Put dialogue and community first, not structure. For instance, make the first thing students see in the LMS the discussion, not your notes or slides, and use the discussion to share content and guide the process. However, I’d mostly feel like the driver of a Model T Ford trying to teach someone to drive a Tesla. Technologies have changed, I have changed, my memory is unreliable.

bicycleIn fact, I haven’t driven a car of any description in years. What I normally do now is, metaphorically, much closer to riding a bicycle, which I happen to do and enjoy a lot in real life too. A bike is a really smart, well-adapted, appropriate, versatile, maintainable, sustainable soft technology for getting around. The journey tends to be much more healthy and enjoyable, traffic jams don’t bother you, you can go all sorts of places cars cannot reach, and you can much more easily stop wherever you like along the way to explore what interests you. You can pretty much guarantee that you will arrive when and where you planned to arrive, give or take a few minutes. In the city, it’s often the fastest way to get around, once you factor in parking etc. It’s very liberating. It is true that more effort is needed to get from A to B, bad weather can be a pain, and it would not be the fastest or most comfortable way to reach the other side of the continent: sometimes, alternative forms of transport are definitely worth taking and I’m not against them when it’s appropriate to use them. And the bike I normally ride does have a little electric motor in one of the wheels that helps push me up hills (not much, but enough) but it doesn’t interfere with the joy (or most of the effort) of riding.  I have learned that low-threshold, adaptable, resilient systems are often much smarter in many ways than high-tech platforms because they are part-human. They can take on your own smartness and creativity in ways no amount of automation can match. This is true of online learning tools as much as it is true of bicycles. Blogs, wikis, email, discussion forums, and so on often beat the pants off learning management systems, commercial teaching platforms, learning analytics tools or AI chatbots for many advanced pedagogical methods because they can become what you want them to be, rather than what the designer thought you wanted, and they can go anywhere, without constraint. Of course, the flip side is that they take more effort, sometimes take more time, and (without enormous care) can make it harder for all concerned to do things that are automated and streamlined in more highly engineered tools, so they might not always be the best option in all circumstances, any more than a bike is the best way to get up a snowy mountain or to cross an ocean.

Why you shouldn’t listen to my advice

It’s sad but true that most of what I would really like to say on the subject of online learning won’t help teachers on the ground right now, and it is actually worse than the help their peers could give them because what I really want to tell them is to change everything and to see the world completely differently. That’s pretty threatening, especially in these already vulnerable times, and not much use if you have a class to teach tomorrow morning.

The AACE book is more grounded in where in-person teachers are now. The chapter “We Need to Help Teachers Withstand Public Criticism as They Learn to Teach Online”, for example, delves into the issues well, in accessible ways that derive from a clear understanding of the context.  However, the book cannot help but be an implicit (and, often, explicit) critique of how teachers currently teach: that’s implied in the title, and in the chapter structures.  If you’re already interested enough in the subject and willing enough to change how you teach that you are reading this book in the first place, then this is great. You are 90% of the way there already, and you are ready to learn those lessons. One of the positive sides of emergency remote teaching has been that it has encouraged some teachers to reflect on their teaching practices and purposes, in ways that will probably continue to be beneficial if and when they return to in-person teaching. They will enjoy this book, and they may be the intended audience. But they are not the ones that really need it.

I would quite like to see (though maybe not to read) a different kind of book containing advice from beginners. Maybe it would have a title something like ‘What I learned in 2020’ or ‘How I survived Zoom.’ Emergency remote teachers might be more inclined to listen to the people who didn’t know the ‘right’ ways of doing things when the crisis began, who really didn’t want to change, who maybe resented the imposition, but who found ways to work through it from where they were then, rather than where the experts think (or know) they should be aiming now. It would no doubt annoy me and other distance learning researchers because, from the perspective of recognized good practice, much of it would probably be terrible but, unlike what we have to offer, it would actually be useful. A few chapters in the AACE book are grounded in concrete experience of this nature, but even they wind up saying what should have happened, framing the solutions in the existing discourse of the distance learning discipline. Most chapters consist of advice from experts who already knew the answers before the pandemic started. It is telling that the word ‘should’ occurs a lot more frequently than it should. This is not a criticism of the authors or editors of the book: the book is clear from the start that it is going to be a critique of current practice and a practical guidebook to the territory, and most of the advice I’ve seen in it so far makes a lot of sense. It’s just not likely to affect many of the ones who have no wish to change not just their practices but their fundamental attitudes to teaching. Sadly, that’s also true of this post which, I think, is therefore more of an explanation of why I’ve been staring into the headlights for most of the pandemic, rather than a serious attempt to help those in need. I hope there’s some value in that because it feels weird to be a (slight, minor, still-learning) expert in the field with very strong opinions about how online learning should work, but to have nothing useful to say on the subject at the one time it ought to have the most impact.

Read the book:

Ferdig, R.E. & Pytash, K.E. (2021). What Teacher Educators Should Have Learned From 2020. Association for the Advancement of Computing in Education (AACE). Retrieved March 22, 2021 from https://www.learntechlib.org/primary/p/219088/.

Evidence mounts that laptops are terrible for students at lectures. So what?

The Verge reports on a variety of studies that show taking notes with laptops during lectures results in decreased learning when compared with notes taken using pen and paper. This tells me three things, none of which is what the article is aiming to tell me:

  1. That the institutions are teaching very badly. Countless decades of far better evidence than that provided in these studies shows that giving lectures with the intent of imparting information like this is close to being the worst way to teach. Don’t blame the students for poor note taking, blame the institutions for poor teaching. Students should not be put in such an awful situation (nor should teachers, for that matter). If students have to take notes in your lectures then you are doing it wrong.
  2. That the students are not skillful laptop notetakers. These studies do not imply that laptops are bad for notetaking, any more than giving students violins that they cannot play implies that violins are bad for making music. It ain’t what you do, it’s the way that you do it. If their classes depend on effective notetaking then teachers should be teaching students how to do it. But, of course, most of them probably never learned to do it well themselves (at least using laptops). It becomes a vicious circle.
  3. That laptop and, especially, software designers have a long way to go before their machines disappear into the background like a pencil and paper. This may be inherent in the medium, inasmuch as a) they are vastly more complex toolsets with much more to learn about, and b) interfaces and apps constantly evolve so, as soon as people have figured out one of them, everything changes under their feet. It becomes a vicious cycle.

The extra cognitive load involved in manipulating a laptop app (and stopping the distractions that manufacturers seem intent on providing even if you have the self-discipline to avoid proactively seeking them yourself) can be a hindrance unless you are proficient to the point that it becomes an unconscious behaviour. Few of us are. Tablets are a better bet, for now, though they too are becoming overburdened with unsought complexity and unwanted distractions. I have for a couple of years now been taking most of my notes at conferences etc with an Apple Pencil and an iPad Pro, because I like the notetaking flexibility, the simplicity, the lack of distraction (albeit that I have to actively manage that), and the tactile sensation of drawing and doodling. All of that likely contributes to making it easier to remember stuff that I want to remember. The main downside is that, though I still gain laptop-like benefits of everything being in one place, of digital permanence, and of it being distributed to all my devices, I have, in the process, lost a bit in terms of searchability and reusability. I may regret it in future, too, because graphic formats tend to be less persistent over decades than text. On the bright side, using a tablet, I am not stuck in one app. If I want to remember a paper or URL (which is most of what I normally want to remember other than my own ideas and connections that are sparked by the speaker) I tend to look it up immediately and save it to Pocket so that I can return to it later, and I do still make use of a simple notepad for things I know I will need later. Horses for courses, and you get a lot more of both with a tablet than you do with a pencil and paper. And, of course, I can still use pen and paper if I want a throwaway single-use record – conference programs can be useful for that.

 

 

 

 

Address of the bookmark: https://www.theverge.com/2017/11/27/16703904/laptop-learning-lecture

Originally posted at: https://landing.athabascau.ca/bookmarks/view/2871283/evidence-mounts-that-laptops-are-terrible-for-students-at-lectures-so-what