The problematic metaphor of the environment in online learning

Jon Dron, Athabasca University, jond@athabascau.ca

Abstract

In online educational systems, teachers often replicate pedagogical methods, and educational institutions replicate systems and structures used by their in-person counterparts, the only purpose of which was to solve problems created by having to teach in a physical environment. At the same time, a great deal of the development and use of learning technologies has focused on creating virtual learning environments that attempt to replicate features of their physical counterparts, thereby weakly replicating in software the problems that in-person teachers had to solve. This has led to a vicious circle of problem creation and problem solving that benefits no one. In this paper I argue that the term ‘environment’ is a dangerously misleading metaphor for the online systems we build to support learning, that leads to poor pedagogical choices and weak digital solutions. I propose an alternative metaphor of infrastructure and services that can enable more flexible, more learner-driven, and more digitally native ways of designing systems (including the tools, pedagogies, and structures) to support learning.

Keywords: online learning, learning environment, learning management system (LMS), Next Generation Digital Learning Environment (NGDLE), personal learning environment (PLE), learning infrastructure.

Introduction

Outside the walls of educational institutions, for those with adequate Internet access, intentional learning using online systems is almost certainly more popular than its in-person counterpart, as at least the first port of call for learning almost anything and, often, as the primary means through which it occurs. From Google Search to Wikipedia, from MOOCs to Twitter exchanges, from YouTube videos to Khan Academy tutorials, people with online access are swamped with  learning opportunities. However, many academics and students still see online education as a poor second-best to in-person learning (e.g. Protopsaltis & Baum, 2019; Bouygues, 2019; Tichavsky, Hunt, Driscoll, & Jicha, 2015). In this paper I will argue that the distinction between online and in-person learning is far less significant than it appears, because all learning is in-person and never online, and most learning that is labelled as ‘in-person’ actually occurs at a distance from the teacher. Problems emerge, however, when  institutional online teaching inadequately attempts to replicate features and forms of in-person teaching, many of which:

1. Exist to solve problems caused by the distinctive physical, temporal, psychological, and economic limitations of material spaces, and
2. Are successful mainly as a result of the features and forms of the physical setting, not as a result of intentional teaching.

As a result, the systems we develop may not take full advantage of the medium, may not take advantage of the physical context of the students, and may attempt to solve problems that should not exist for those using them to learn because they are the result of in-person constraints. In many cases, online teaching may therefore actively militate against effective learning. Some of the problems may be solved using pedagogical adjustments in teaching and organizational changes at an institutional level, discussed in the first part of this paper. However, many emerge from the electronic systems that we use to teach, that poorly mimic the functions of their in-person counterparts in software. In this paper, I suggest that this is, to a significant extent, due to the misappropriation of spatial metaphors that cannot and should not be applied to online systems. I propose a different approach to the construction and conceptualization of tools for online learning, that better reflects the innate benefits of the medium, and that more fully supports the needs and circumstances of both online students and online teachers. I conclude by putting the pieces together and suggesting ways that, in combination, pedagogical, organizational, and digital changes may co-evolve to achieve the potential transformation of education that is afforded by digital networked devices.

In-person teaching

Although they may have physical or virtual windows to the world outside, the walls of the classroom provide clear boundaries that define the space and, because participants must be co-present,  the time in which activities intended to bring about learning occur. This is also true of most of the other buildings, rooms, and spaces that are provided by in-person institutions for students, including corridors, student accommodation, meeting rooms, common rooms, cafes, halls, quadrangles, staff offices, libraries, gymnasia, and examination halls. These spaces are not just support structures for the classroom, but active participants in the learning process (Dron. 2021). Even the act of physically walking to the classroom, especially with other people, creates a salience and value to the activity that is very different from that of clicking a link to an online resource. Most significantly, they are social spaces where learning happens as a result of direct and indirect interactions between learners (who are one another’s teachers) and, often, with their designated teachers. Simply seeing others learning makes a difference, as do the fliers and leaflets on the walls, and the spaces intended to support clubs and societies, where much academic discussion often occurs. Cafes, bars, and canteens are rich in learning dialogue. Student rooms, dormitories, and (especially) their kitchens, are powerful seed beds for learning, where much sense-making discussion occurs. Many universities provide purpose-built study areas. Even and perhaps especially, areas cordoned off for smokers provide an extremely fertile space where students from different subject areas and disciplines can and do share and construct their knowledge. Similarities in their design the world over speak to the fact that these are highly evolved spaces, supporting a learning process that extends far beyond the classroom.

These countless diverse learning opportunities in physical spaces, perhaps counter-intuitively, speak to the fact that there is a distance component to virtually all in-person education. Indeed, almost all learning is distance learning, in the sense of occurring somewhere and somewhen other than where and/or when deliberate, instrumental teaching occurred. People who learn with teachers in a physical space are almost always also interacting with other participants in the teaching role at a distance, usually in time and space, such as textbook authors, classroom designers, editors, illustrators, creators of timetables, and curriculum designers. And, for ‘in-person’ institutional learners, much of the learning itself also occurs at a distance, outside the classroom. This is most obvious in the form of assignments and homework but just as much learning can occur in conversation and interaction with others. Even when alone, if teaching works, sense-making connections always occur after the lesson is over, and continue to do so long after (sometimes decades after) the teaching event, almost never in the same place that the lesson originally occurred. In-person students do not have one teacher: they may have thousands. Weaknesses in in-person teachers can often be compensated for by these many other teachers, including the learners themselves and the institutions that provide the framework and resources for learning. This is amplified by that fact that, although credentials and grades are highly antagonistic to persistent intrinsic motivation (Kohn, 2011; Blum & Kohn, 2020; Ryan & Deci, 2017), they do encourage compliance. In search of good grades, students will therefore make use of whatever means they can – including those many other teachers as well as cheating or satisficing – to achieve the marks they seek. The physical environment of an in-person institution provides many supports to make this possible.

Online teaching

The in-person teacher, by default, controls learning in the classroom because it is a self-contained environment of which they are in charge for the duration of the lesson. Relinquishing control must be an active choice, or the result of an error. In contrast, the online teacher cannot, without a great deal of concerted effort, control the online student, any more than a writer of a book can control a reader. Online students can always choose when, where, with whom, how, for how long, and with what tools, media, and resources they learn (Dron & Anderson, 2014). It would therefore be surprising were online pedagogies to closely resemble their in-person counterparts, because they have different problems to solve. Most notably, without the requirement to share a single environment, with all the many rules, norms, structures, and constraints that entails, and without the need for the teacher to fill every moment of classroom time with learning activities, there should be no need for teachers to exercise the same level of control over their students.  However, online teaching evolved from in-person teaching, and online institutions must continue to interoperate with the in-person educational systems of which they are a part. As a result, many online teachers assume that they should dictate the learning process as much as their in-person counterparts and, usually, it becomes a partly self-fulfilling assumption through coercive methods like frequent grading, draconian scheduling, and tests. They consequently often make use of very similar pedagogies to those of their in-person colleagues, struggling to find simulacra or workarounds for the affordances of physical spaces that are no longer available, vainly believing that the learner is going to follow the path that they have determined for them and, too often, imagining that this is the sum total of the learning experience. To make matters worse, educational institutions impose other structures that are purely the result of constraints of teaching in physical classrooms, such as fixed-length (or multiples of fixed lengths) courses, deadlines, and perhaps most perniciously, the concept of failure. As any game-player or musician knows, failure is part of learning: it cannot ever be its end but, because of the constraints of having to run a course with co-present students and a beginning and end, failure becomes a potential outcome, not just part of the process. When all these factors are put together, the online student may have little more independence than their in-person counterpart but, at the same time, may lack the countless structures and forms of physical institutions that support in-person students. Rather than being immersed in learning opportunities, they must actively seek them within their own physical and virtual environments.

One obvious solution to this problem would be to create an online learning system that provides much of what is lost in translation from the physical environment, for example through a custom-made social media platform or an informal discussion area within a learning management system. However, this is not as easy or effective as it may seem, especially if it remains tightly coupled with other institutional policies, norms, and teaching methods. Partly, this is because of the too common focus on explicit outcomes and grading found in most institutional teaching together with failure by students and teachers to recognize the critical role of in-between spaces in learning. Thanks to the extrinsic coercion of marks and credentials,  if it makes no direct contribution to a grade, then it is seen as less valuable. Mainly, though, it is because it is not just there: students will not pass it on their way to somewhere else or be there for other reasons (like a need for rest or refreshment). They have to intentionally visit, typically with a purpose in mind. However, as the main value of it is its purposelessness (or, at least, that it supports a very broad set of purposes), that is rarely going to happen. Online systems are not environments in which students dwell: they are parts of their own environments.

This speaks to the central phenomenon around which this paper revolves: that nobody actually learns anything at a distance. We are always learning it where we are now. All learning is in-person learning, and it all takes place within a physical environment, part of which (but only a part) may include whatever technologies we might be using to talk with people, read, watch, listen, and learn from: books, computers, pens, emails, learning management systems (LMSs) and so on. Some of these may extend into other physical spaces occupied by other people, perhaps at other times, connected by online means. The broader learning environment is highly distributed in time and space, but learning itself only occurs locally. What we describe as ‘distance learning’ or ‘online learning’ is thus, in fact, nothing of the kind. It may involve distance or online teaching but the learning is always in-person. Online students exist, because the word ‘student’ only has meaning in relation to an online teacher, but online learners do not.

The promise of online learning environments

It is understandable that, when we teach in person, we have to occupy and make different uses of the same or similar environments like classrooms, labs, workshops, lecture theatres, and offices. There are huge financial, physical, and organizational constraints on making the environment fit the task, so it would normally be madness to build or even to substantially reconfigure a whole new classroom every time we wished to run a different class. Rooms may be built for flexibility, with moveable partitions and furniture, and that is much to be wished for, but there are physical limits such as walls and property boundaries that prevent this from going too far. Instead, our pedagogies and processes are normally made to fit the affordances and constraints of the classroom: they are another problem that our pedagogies have to solve, and/or an opportunity that our pedagogies can take advantage of. We may, sometimes, have some choice between classrooms that offer different facilities but, for the most part, our options are limited by what has already been built.

Online, there are countless tools available and, if none are suitable, it is not too hard to build them or to modify them to suit our needs, at least when compared with the costs of creating new physical spaces. There are few significant physical limits on how many can be used or how many people may use them: there are none of the limitations of physical space that constrain the use of buildings. Once they are built, moving between virtual tools just takes a tap of a screen or the click of a mouse or keyboard. It is even possible to use several of them at once, especially with a large high resolution monitor or more than one device. The learner’s environment may contain countless tools and systems, any of which may support learning, including physical books, instruments, and other people around them. And yet, for the most part, online teachers tend to make use of only a handful of possible tools: most consist of no more than a learning management system , email, and perhaps a webinar system.

There are many mutually reinforcing reasons that online teachers rarely provide the perfect application or combination of applications for the context of study:

• Teachers’ lack of knowledge of the options (it takes time and effort to discover what’s available).
• Teachers’ lack of skill in using them (most interesting tools have a learning curve, and that gets steeper in inverse proportion to the softness and diversity of the toolset, so most teachers don’t even know how to make the most of what they already have).
• Lack of time and/or money for development (an application is just a shell for the content it contains and the interactions it supports, and it is not always as easy to add existing materials to a new tool as it might be in a physical space: for example, an in-person lecturer only needs to talk, whereas an online teacher must master the complexities of the hardware and software needed to record, edit, and share the same thing ).
• Costs and difficulties in management (each tool adds costs in managing faults, configuration, accounting for use, performance, and security).
• Cognitive load involved for learners in adapting to the metaphors, signposts, and methods needed to use the tool itself.

All of these are a direct consequence of the very diversity that would make us want to use different applications in the first place. This is a classic Faustian Bargain (Postman, 1998) in which the technology does what we want, and in the process creates new problems to solve.  Every digital system must establish rules of engagement that its users must learn, such as the ways that navigation occurs, the ways to make it perform its functions, the terminology it uses, and so on. In effect, every application invents its own metaphorical physics. That makes virtual systems harder to find out about, harder to learn, harder to develop, costlier to manage, and more difficult to navigate than the static, fixed facilities found in particular physical locations. They are all different, there are few if any universals, and any universal today may become a conditional tomorrow. In the case of cloud-hosted systems, the owners of which may unilaterally make changes to the software or configuration, this may be literally so.

Learning management systems

The learning management system (LMS) addresses all of these problems, to some extent. Almost every LMS essentially automates the functions, though not exactly the form, of traditional classrooms. Indeed, they are typically seen as environments, or are referred to as ‘platforms’, underlining the physical metaphors that inform them. In some parts of the world people prefer to use the term ‘managed learning environment’ (MLE), and the LMS/MLE is, in most vocabularies, the most dominant representative of a larger category of systems usually described as virtual learning environments (VLEs) that also includes things like MOOs (multi-user dungeons, object oriented), immersive learning environments, and simpler web-based teaching systems that replicate aspects of physical teaching such as Google Classroom or Microsoft Classroom. The use of spatial metaphors for the names of such systems reflects a deep-held belief or tacit assumption that the virtual systems can provide the boundaries within which actions occur, in ways that tend to be seen as analogous to those of physical spaces. In a few limited contexts, notably through immersive systems, this belief may be partly justified,. However, it matters that even the most immersive system occurs in a physical space. For instance:

• when participants leave the immersive system they exit into different rooms, losing the natural opportunities for incidental or continuing chat that are innate to physical spaces;
• participants are at the mercy of dropped network connections, glitches, and issues with the machines that run the immersive environment, leading to potentially quite different experiences for different participants;
• different participants experience different temperatures, background sounds, smells, and opportunities for interruption in their own physical environments.

LMSs differ from physical environments to a much greater extent than immersive systems, in ways I will describe over the rest of this section. This is not a trivial issue of nomenclature. I will be arguing that the misconception that they are meaningfully analogous to physical classrooms lies at the heart of many weaknesses and failings in both the design of the tools and their use, reinforcing the belief that online teaching closely resembles in-person teaching, and blinding us to essential differences between the two.

The building metaphor

Creators of early LMSs and VLEs back in the 1990s (including the author) based their designs on the functions and entities found in a traditional university because that was the context from which they sprang, and that was the context in which they had to fit. In the eyes of its designer, an LMS could be thought of as a big university building with rather uniform classrooms. It may have extensions built onto it using plugins or standards such as LTI (the learning tools interoperability standard), and it may have a few doors and gateways (mainly in the form of hyperlinks) linking it circuitously or in jury-rigged fashion to other similarly weakly connected ‘buildings’  such as ‘places’ to register, to seek support, to talk to an advisor, to complain, to find books, and so on. For the most part, though, its fundamental organizational metaphor is that of a university, college, or school.

 

The LMS is, however, an impoverished school. It has no metaphorical corridors, halls, common rooms, canteens, yards, libraries or any of the other parts of a typical university environment where students gather to (amongst other things) learn. Students rarely get to even be aware of other classrooms beyond those they are in. Some teachers may give classrooms informal-sounding names like ‘the learning cafe’ but it is still just another classroom that works in the same way as the rest. Students teleport from one classroom to the next because what happens in between is not perceived by the designers as a useful classroom function to be automated or perhaps, more charitably, they could not figure out how to automate that. But there are other differences that are, perhaps, even more pernicious, to which we turn next.

Centralized code bases

In a physical environment, every object is discrete, occupying its own space. Physical classrooms can and do change – new furnishings, equipment, and so on – but the effects are local to that particular space, and they seldom prevent teaching from occurring across an institution.  Learning management systems, on the other hand, re-use the exact same code to generate all of the virtual classrooms of which they consist. Instead of a number of courses occupying the same physical spaces, and there being many such spaces to choose from, every course gets its own instantiation of a single centrally hosted toolset. There may be options to switch features on and off within any given course instantiation, options to configure each component differently, and a choice between components may be offered, but everyone gets exactly the same set of features, determined by the developer and the system administrator. This means that one set of features has to suit everyone. If, say, a teacher wants a discussion component that does things the default discussion component does not support, then it has to be installed or integrated in the centralized code base. While the LMS may technically support this – through plugins, LTI integrations, OKI components, and so on – system administrators are usually rightly reluctant or unable to allow it. Every component is another potential source of failure or (often) security holes, incurs management costs, uses system resources, creates a significant maintenance burden, and increases the complexity of the system for everyone. To allow unfettered installation of alternative components would be completely unmanageable. As a result, most available features must be a compromise, that can be bent to suit the needs of (typically) thousands of courses and teachers, but that are unlikely to be an ideal fit with any of them. Unlike the physical classroom, changes to the underlying application affect everyone, at once. When the LMS goes down, it takes the whole institution with it, and when changes are made, they are made for everyone, often affecting hundreds or thousands of courses and tens of thousands of students.

This is particularly problematic in cloud-based systems where administrators are not even part of the same organization, and where the system must support hundreds or thousands of institutions. Few of us who teach using cloud-based systems have not experienced difficulties when the systems on which our courses run change without warning or consultation, disabling or altering things that disrupt the design, sometimes rendering it inoperable. Even when they work, the fact that they use a single code base limits the potential for customization. Because most LMSs based their designs on what was presumed to occur in an average university, they rarely fit well with any actual university, because virtually no universities are average. Sometimes, the problems may be relatively minor. For example, Blackboard calls its organization elements  ‘courses’, whereas many other names for such things are common, including modules, units, and papers, and ‘course’ may refer to what others around the world might call a ’program’. Even this may disrupt and cause confusion (Dron, 2006). Other problems can run deeper, to which we turn next.

Reified roles

The typical LMS is a very controlled environment where everyone has a programmatically enforced role (typically at least partially reflecting traditional educational roles), that may vary according to the ‘room’ in question, but that are far less fluid than those in physical spaces. There are strong hierarchies, and limited opportunities for moving between them. Some of those hierarchies are native to the online learning system: the system administrator, for instance, has far more power than anyone in a physical university to determine how learning happens, like an architect with the power to move walls, change the decor, add extensions, and so on, at will. The programmers of the system are almost god-like in their command of its metaphorical physics. But the ways that they give teachers (or learning designers, or administrators) control, as designers, directors, and regulators of the classroom, are perhaps the most pernicious. In a classroom a teacher may lead, and that is the default, but they may and usually should choose to at least share leadership with their students, often fluidly and in response to how students are learning. In an LMS, a teacher (or someone playing that role) must lead,

Tools such as discussion forums may seem to be more egalitarian, but teachers’ power to control events in them is usually far greater than that of their in-person colleagues, often including the means to delete unwanted messages, prevent replies, stop conversation threads stone dead, and many other things that would be superhuman capabilities in a physical space. In a physical classroom, a determined enough student can always make themselves heard. In an LMS, the teacher can silence them. There is thus less of the soft flexibility found within in-person classrooms that allows for conversational pedagogies that adapt to the interests and needs of learners. At the same time, though, it should be (though too rarely is) remembered that the teacher’s power is confined to a small part of the learner’s own environment, not to a whole classroom. In practice, teachers still tend to treat the forum as an analogue of the classroom and, recognizing the value of dialogue in such contexts, often resort to coercion to make it happen online: marks for discussion contributions are far more common than in in-person settings, even among experienced online teachers. This combination of hard, role-based digital authority and hard, reward-based pedagogical authority is fundamentally different from its physical analogue. It creates both a social and a power distance that compounds what is already a less immediate relationship between student and teacher.

Within the LMS the teacher sees things that students cannot, and controls things that the students may not. A teacher configures the space, and determines with some precision how it will be used. With a lot of effort and (usually) high risk to the security and stability of the system, it can be made to behave differently, but it almost never is, because doing so usually involves promoting students to roles with similar capabilities to that of the teacher. In many cases, especially when it involves the use of plugins or other tools that extend across the system, this cannot be localized, so the risks to every user of the system must be considered. This is beyond the capabilities or rights of most teachers, and so it usually falls to system administrators, reinforcing their already substantial power to affect the teaching process.

Functional design

An LMS is typically built along functional lines. Rather than attempting to be a precise mirror of the in-person context, its functions are mostly based on loose, superficial observations of the things that teachers and students seem to do in physical classrooms, analysed to their component parts. Mostly, they are structured by teaching functions: presenting, discussing, assessing, guiding, and so on. For instance, in most LMSs, if you want to talk with someone, you normally need to go to a separate discussion area inside the classroom or, metaphorically,  to leave a note on the teacher’s desk in the form of a direct message. Unlike a physical classroom, dialogue is seldom possible everywhere. The same is true if you want to take a test, or to share your work with others: it rarely occurs within the context of learning, but in a separate screen, often separated from its context by a hierarchical set of links. Indeed, in many architectures, it will be handled by a different component than the rest, with its own tables in the database and its own distinctive interface.

Similarly, lectures are either literally that (video recordings of lectures) or (more usefully, from a learning perspective), text and images to be read on screen. This results from the erroneous assumption that the only function of lectures is information transmission, which is perhaps their least useful role, given that we have known for almost a century that it is far more effective to read a book (Greene, 1928). Lectures can and do have value as physical and temporal signposts, as motivators to pay attention, as events that demand attendance and thus have greater salience than simple reading, as well as providing opportunities to engage with others, sometimes within but always outside the lecture hall. Online, there is seldom a chance for students to even put up a metaphorical hand to question the teacher, and ‘joining’ a lecture is no more salient than clicking a link to a Facebook post. There are limited opportunities to be aware of what other students are doing, including for the teacher (although teachers do usually have access to system logs that offer an impoverished caricature of what students are doing, albeit one that is blind to anything they do beyond clicking and tapping keys on a machine). Much of the ‘space’ may as well be unpopulated, given the little students see of one another. Learning resources are normally static and designed in advance, and so the teacher cannot nimbly adjust to student reactions to them. Notices can usually only be pinned on the ‘wall’ by teachers, often with names such as ‘announcements’, further emphasizing the controlling nature of the teacher-student relationship. Classroom timetables are embodied in software despite the fact that a rigid and unforgiving timetable makes little sense in a medium that supports learning anywhere, any time. Some LMSs may allow you to break up the content differently, but it is still another timetable; just a timetable without dates. It is always the teacher (or one to whom the role is delegated) who sets the order, pacing and content.

Robot overlords

The LMS provides a high-tech classroom, populated by metaphorical robots.

Some of the robots may be programmed to attempt to force students to behave in ways determined by those higher in the hierarchy (sometimes teachers, sometimes administrators, sometimes the programmers of the software). For instance, adaptive systems might act as gatekeepers that prevent students from moving on to the next section of work before completing the current one, or they might prevent students from submitting work before or after a specified date (Martin, Chen, Moore, & Westine, 2020), or they might limit their access to a specified time period.

Some of the robots might even mark your work (Keuning, Jeuring, & Heeren, 2018). Human beings have grown up with other humans and therefore understand the context of the work, the motivations of the students, and the many different ways that things can go wrong, as well as creative and unexpected ways they can go right. Robots – even those that are employ deep learning and similar AI approaches – do not. While hard, mechanistic systems may be useful for providing feedback when students must play their role correctly in hard, mechanistic systems (in hard, ‘right answer’ subjects), those mechanistic skills are seldom the most important part of what they learn. Human teachers do not (or should not) just judge success or failure: they should model practice,  remedy misconceptions, provide encouragement, and so on.

There are metaphorical surveillance cameras everywhere, recording students’ every move (in very low resolution), often only accessible to those with more powerful roles, though sometimes a robot or two might give them a filtered view of it, such as through learning analytics traffic-light interfaces (Verbert, Duval, Klerkx, Govaerts, & Santos, 2013). Though the perpetrators of these tools may claim to have student interests in mind, and will often talk of ‘personalization‘ by way of justification, it is not personalization at all: it is system-enforced customization done to, not by the students (Kohn, 2015). These are all tools that are designed to enforce compliance: an attempt to embody in software the control that is demanded of an in-person teacher due to an accident of physics, not for any pedagogical purpose.

Beginnings and ends

The fundamental social form of the classroom that provides the primary metaphor of most LMSs is the formal group (Dron & Anderson, 2014). Formal groups are technological entities – inventions that are designed to address problems – at least as much as they are social. Among their many technological features are names, roles, procedures, rites of joining and leaving, rules of behaviour, schedules, beginnings and ends, almost all of which arise from the constraints of in-person learning, such as the need for people to be co-present, problems when people talk at once, limits to the capacity of classrooms, directionality of hearing and sight, and so on (Dron, 2016). Unsurprisingly, many of these features are embodied in code, not only in the reified roles already discussed but in processes of joining and processes of leaving.

A student cannot usually go back and visit when their course is over because most online courses have opening and closing enrolment dates. Perhaps their designers assumed that, when teaching was done, the learning was done which, of course, it never is. Learning keeps on evolving long after explicit teaching and testing occurred. Again, this is because physical classes are scheduled and terms come to an end because they must, not because it makes pedagogical sense. And, like almost everything, it is possible to override this default, but hardly anyone ever does, partly because it brings back those Faustian bargains, especially in manageability, but mainly because most people accept defaults (Kelly, 2009, Dron, 2006). LMSs embody enrolment technologies as much as they do teaching technologies and, in the process, they unnecessarily limit potential for learning.

Because the primary metaphor of almost all LMSs is the classroom, they can be a particularly poor fit with ways of teaching that have no classes, such as self-paced courses and MOOCs, individual projects, or flexible networked ways of learning such as those underpinned by Connectivist, or Rhizomatic models of learning. This is not to say that such uses are impossible. For example, assumptions about class schedules that are embedded in software (such as that all students must submit work by a certain deadline) can be disabled, or bypassed by setting a deadline in the far distant future, then manually informing students of when to submit their work. However, the fit with self-paced models of learning is typically poor. Among the many peculiarities that result are students who engage in discussions with ‘classmates’ who no longer have access to the provided forum, and the impossibility of collaboration when every student is at a different point in the course. More challengingly, and unlike teacher-paced courses in which the teacher can modify almost any aspect of the content or curriculum at will, knowing that the whole class will be affected in the same way, much confusion and even dismay can arise when changes are made to materials that may be in use by existing students.

Imperfect caricatures of physical spaces

In summary, most LMSs provide an automated set of metaphorical classrooms that harden many of the undesirable side-effects of educational systems in software, in ways that have little to do with how best to teach, and that inappropriately apply spatial metaphors in ways that conceal rather than illuminate their functions. Each bit of automation and each navigational decision hardens pedagogical choices, at least as much as the walls, doors, and physical limitations of physical spaces and, often, more. Programmers do not replicate physical classrooms but instead create or enlist new laws, new kinds of structure, and new kinds of hardened process that can be embodied in code. Classrooms solved problems of physics for in-person teaching and form part of a much larger structure that has evolved to teach reasonably well. LMSs just focus on a limited subset of teaching roles, and empower the teacher in ways that caricature their already excessive dominance in the classroom, that only occurred because of the nature of the physical space and the constraints it imposed.

LMSs leave much to be desired, but the metaphors on which they are based bear enough resemblance to physical reality to be readily understood by teachers and students. They usually provide just enough configurability and flexibility to more or less adequately work as teaching tools, for everyone, almost no matter what their level of digital proficiency might be. They more or less address the Faustian bargains listed earlier, albeit they normally do so by stifling what we wanted and should have been able to do in the first place with online tools, In the process they create new and quite extensive problems, as well as failing to replicate most of what makes physical universities work in the first place. Virtual learning environments are not like physical learning environments: they are only ever parts of them. There are other electronic ‘places’ to escape from them, such dedicated social media, or even just plain old email, but then all those Faustian bargains come back to haunt us again. They occupy space within the learner’s own physical environment, but it is rare for pedagogical designs to even acknowledge that, let alone to consider it in the design.

Improving the LMS

It is tempting, faced with these problems, to assume that they could be solved if only we made the LMS more closely resemble the physical environments on which it is modeled. However, this is a poor solution because, as we have already seen:

1)    physical environments create constraints and problems to solve that are unnecessary and avoidable in virtual systems; and

2)    it is not practical nor is it within our technical reach to replicate all the many incidental benefits of physical environments.

That said, there are lessons to be learned from physical spaces. Among many improvements that could be made would be:

1)    To make every part of the system at least potentially social: to allow synchronous and/or asynchronous dialogue to occur on every page or screen of the system. This is the default in all physical spaces: talking has to be prohibited if it is not wanted.

2)    To allow at least some parts of the system to be free of roles, or with more flexible roles, allowing all members of the system to create and share posts and resources using discretionary access control (so it is the poster’s responsibility to choose who can see it, and who can change it). Even in highly controlled physical environments, we choose what we reveal and to whom.

3)    To support social networking and the blurring of boundaries between areas, tools, and features of the site, so that courses are just one of many kinds of organizational unit, with selectively permeable boundaries through which others can pass, or with which they can overlap. Again, this is a default in physical spaces, that leak information through walls, floors, windows, and doors, that exhibit continuity of engagement when people enter or leave classrooms, that allow teachers to open doors to others, that admit a multiplicity of primary uses.

Though these improvements appear simple to achieve, adapting an existing mainstream LMS such as Blackboard, Canvas, Moodle, or BrightSpace to support them is fraught with difficulty.

By far the easiest of these improvements to make within an existing LMS is to make it more social. Achieving this within an existing course structure is a simple programming problem that can readily be solved in countless ways. In most LMSs, it could be built as a plugin. Existing architectures, in which courses and roles play a primary structural role, make it somewhat more difficult to extend such dialogue beyond the boundaries of the course. The metaphorical walls of a course are, for the most part, more of a barrier to engagement beyond it than those of a physical classroom because their metaphorical physics can be (and is) enforced in code. It is not, however, an insoluble problem. For example, a context-aware embeddable discussion system such as Disqus or Isso, hosted locally or remotely, could fairly easily be added.

Making the system free of roles is much more difficult because, in most LMSs, they underpin almost every function and structure of the system, and they cannot be made to work with an open, discretionary access- based model of permissions: the two approaches are, architecturally, mutually exclusive. One way of dealing with this would be to follow the lead of the Drupal content management system to support ‘organic’ groups: limited areas of the LMS where everyone has the same rights to create shared content or social areas, and where anyone can control who can see what they post. These areas could be as large or as small as desired but it would be difficult to make them extend beyond a course, or to encompass one or more courses. It would not be impossible, but to do it safely and reliably (without giving everyone a single, very powerful role) would require a major rewrite of the underlying LMS.

For all of the LMSs of which I am aware, the most difficult of all these improvements would be to blur the boundaries of the tools, features, and courses. The course is such a fundamental architectural unit of most, if not all, LMSs that changes to its operation would demand a significant redesign. It could be done, but it would not be the same kind of system any more.

It is for these reasons that, wishing to support all of these features and realizing the extreme difficulty of modifying the LMS without compromising some or all of its existing functionality, a group of us at Athabasca University created The Landing (Dron & Anderson, 2014), as a separate system to the LMS, linked only by a single sign-on and tenuous hyperlinks and, to a limited extent (only supporting public posts in either direction) RSS feeds. Further efforts to design deeper integration proved too difficult, for both technical and organizational reasons. Unfortunately, The Landing suffers from the same Faustian Bargain that besets all attempts to expand the range of systems available. The maintenance burden of a system with many thousands of users is too much to sustain for a system with very limited central support and even more limited funding. Pedagogically, the system fulfills an important need and so it has survived for more than 12 years but, technically and from a management perspective, its future is in jeopardy. Similar issues are playing out the world over. The more control and diversity that we enable, the more difficult and expensive it is to manage it.

Alternative approaches

Incremental improvements

Athabasca University is currently building an Integrated Learning Environment (ILE) that centres around very conventional elements of a institutional teaching system: an LMS, some relationship management tools, a student records system, an enrolment system, an examination management system, and so on. These are tightly integrated, but it is intended that the ILE will also embrace many other tools and systems that are far less institutionally bound, from the aforementioned Landing, to other social media (such as WordPress), to portfolio tools, to shared software repositories. This is an approach that starts with replicating existing structures and services by building a tightly managed administrative core, but that is intended to grow to support more open, diverse, and rich approaches to learning and teaching, co-evolving with methods and pedagogies that are more in keeping with the different problems and needs of distance learners. However, though it provides a managed approach to supporting change, this approach carries many risks.

A design approach that treats online systems as environments invariably makes the assumption that it is where everything associated with what goes on inside it happens, and (for online systems) this creates quite unnecessary restrictions on what can happen. Athabasca University’s design approach for its ILE was highly participative, engaging most of its teaching, technical, and administrative staff and asking for what they needed. However, inevitably, their requests were based on assumptions formed by their existing practices and, especially, by the existing environmental metaphors of the LMS and associated systems with which they were already familiar. In essence, they were asked what kinds of spaces they needed, and what kinds of stuff needed to be in those spaces for them to do what they currently do.  ‘Space’ and ‘stuff’ are what Stewart Brand (1997) describes as inevitably being the fastest-changing, most volatile parts of any physical building, after site (its physical limits), structure (what holds it up), skin (mainly the external walls), and services (electricity, gas, network wiring, etc). More abstractly, this is a solid structural principle that applies as much to ecosystems and educational systems as it does to buildings. As Brand himself observes, drawing from O’Neill, DeAngelis, Waide, & Allen (1986), the larger, slower-changing elements of any system affect the smaller, faster-changing more than vice versa. In physical spaces, these naturally tend to be bigger and/or more difficult to change, but the same is true in virtual spaces, where size seldom matters that much, but hardness (inflexibility, brittleness) has the same effect. The more difficult it is to make changes, the more an element of the system determines the behaviour of other elements in the system that interact with it. The ILE’s structure, skin and services have been designed based on needs determined by perceptions of the space and stuff within it, that were in turn very strongly determined by the LMS and other systems that went before, with all the inherited baggage that they inherited from in-person environments. Hence, the ILE’s fundamental design model is really no more than an extended LMS, and it inherits most of its weaknesses. The main way in which it differs is that it is designed to be extendable, but those extensions will still – in terms of how they are treated and used – be part of that same environment, with all the aforementioned problems that this entails.

Integrated learning infrastructures

I have argued that a better name for the system being developed at Athabasca University is not an ‘integrated learning environment’ but an ‘integrated learning infrastructure’ (ILI). In metaphorical terms, it should be like the utilities, services, and mechanisms that make an environment possible, but it should never be thought of as the environment itself.

Stripped to their essentials, digital systems intended to support the educational process provide services, consisting of tools that may be used to support learning, teaching, accreditation, and other roles and functions of an educational system.  Such services are many and various: discussions, presentations, file sharing, assignment submission & grading, quizzes, blogs, scheduling, wikis, bookmarking, real-time communications tools, enrolment systems, identity management systems, support systems, and much more. There is no good reason that these should be confined to loose approximations of their physical counterparts, nor is there any good reason that teachers or system administrators should be the only ones to control them, though it is important that each of them is owned by someone, otherwise the resulting free-for all would be difficult to manage. Microservice architectures that support such systems are quite mature, and widely implemented in different fields, if not so much within the educational sector. From the point of view of end users, these can be thought of as assemblable components, and the assemblies can be performed by anyone, including students. Ideally, it should be possible to integrate them with other applications and services offered beyond an institution, including on the desktop of individual students.

Ideally, it should be possible to assemble them into units with value in the system, that can themselves be assembled into other components. This provides a path for evolution from existing approaches because those units might include courses. There may be a need for additional services to support non-teaching functions associated with educational systems, such as administration or credentialling.

Such services are not so much environments as they are infrastructure that exists within and between the different environments that learners, teachers, administrators, and technicians occupy both virtually and in person. Non-exclusively, such infrastructure may minimally support needs such as:

• Dialogue and interactions between participants
• The presentation and curation of content
• Assessment, formal and informal
• Sharing of words, images, video, audio, and other document types
• The formation of groups,  networks, and sets (social gatherings around shared interests or other commonalities)
• Sharing of tools and resources
• Etc.

What matters most is that all of these services can be combined in indefinitely many ways, by anyone.

This is not a new idea. In the early 2000s, the ELF (e-learning framework) and OKI (Open Knowledge Initiative) both attempted to provide ways to assemble services (ELF) or components (OKI) in many different ways. However, for the most part, both of these initiatives were firmly focused on building centralized systems that replicated the functions of an LMS, so they carried forward the assumption that what would be built from the components would be teaching environments; a better LMS, but still an LMS.

Around the same time as ELF and OKI were being developed, and driven by similar intents,  the notion of the personal learning environment (PLE) became popular, though with very many quite radically different interpretations (Martindale & Dowdy, 2010), ranging from institutionally controlled systems that were often described as ‘platforms’ (Yen et al, 2019) to collections of applications and services assembled by a learner on their own desktop in an ad hoc fashion (Wilson, 2008). Though some of the promoters of the concept saw the environment as extending beyond virtual systems, the vast majority of these interpretations considered only the digital tools, not the physical and social environment of the learner, nor the pedagogical and technical skills used by learners to create and manage those tools. Again, the ‘environment’ metaphor was inadequate and misleading. The PLE was also, for the most part, a concept, not a technology, though efforts were made in some circles to create standards for mashing up those tools, most notably through work on ELF which was, by some, seen as the VLE of the future (Wilson, 2005, cited in Martindale & Dowdy, 2010), and a number of systems were built that were described as PLEs, but that were essentially another kind of institutionally managed server, much like Athabasca Landing, referred to previously.  A more promising set of standards that did focus on the development of standards-based widgets that could be assembled by individuals as well as within an LMS or other system (Wilson, Sharples & Griffiths, 2008), failed to gain enough momentum, despite endorsement of the widget specification by the W3 Consortium, and implementations within all major operating systems. Meanwhile, the term ‘PLE’ itself became such an amorphous concept that even conversations about it were difficult to sustain, let alone useful implementations.

In more recent years, the Educause organization has vigorously promoted the Next Generation Digital Learning Environment (NGDLE), which is essentially very similar in purpose and approach to the earlier ELF initiative, but that:

1)    Takes into account the possibility of learners assembling their own digital toolsets;

2)    Incorporates developments in analytics and artificial intelligence, and

3)    that is largely agnostic to standards used for its implementation, although it does recommend standards and protocols such as xAPI, LTI, learning record stores, and Caliper to help bind them together (Brown, Dehoney, & Millichap, 2015).

Combining the best ideas from service-based systems and work on PLEs, the initiative shows promise. While, once again, the ‘environment’ metaphor fails to extend into the actual spaces that it is intended to be deployed, the initiative is a genuine move beyond the teacher-centric, classroom-inspired models of the LMS and towards a student-oriented service-provision model. There are now some implementations of the concept. For example, the OERu aggregates a wide assortment of open source tools systems providing services such as discussion, microblogging, blogging, wikis, social bookmarking, and so on, that can be used independently by students or as part of the university’s own system (Lane & Good, 2019). While these are still largely perceived as an environment composed of environments, the potential for such a design approach is to free us from the traditional classroom metaphors of the LMS.

Institutional teaching beyond virtual environments

A distance learner’s environment is never digital, though digital tools and services can comprise important parts of it. A learning environment is not just comprised of physical or virtual structures but also the social, pedagogical, organizational, personal, and other dynamic elements that determine how the parts of the structure evolve and interact. It is not just physical matter, or virtual systems, but also the people and what they do together. It is not just how teachers teach, but how learners teach themselves, and teach one another, and are taught by the countless teachers who create the websites, interactions, tools, and structures of the broader internet, and the many teachers who inhabit their own physical spaces, from family members to people in the street. How, therefore, should teachers in institutions teach, when they are just parts of someone else’s environment, co-players in the process, and what kinds of digital tools and systems will be needed to support that?

Perhaps one of the reasons that it is too easy to fall into the trap of thinking of the digital tools and systems as an environment is that it they are  an obvious class of things around which to put a boundary. However, an infrastructure is not just the digital tools but also the human-enacted methods, rules, protocols, and standards that accompany it. It is not just what we use, but the ways that we use it.  It is natural to focus mainly on the software and hardware when designing an online system to support learning, and thus to come to think of it as providing the learning environment itself. If, instead, we remember that we are only building tools to use in the learner’s own environment, and that we are just providers or curators, not controllers or managers of that environment, then a critical and oft overlooked design principle becomes clear: that online students are the primary orchestrators of their learning rather than, as in the physical classroom, their teachers.

An integrated learning infrastructure should therefore not attempt to replicate the form and structure of a traditional classroom, nor should it solely support teachers in assembling the tools needed for their teaching. Instead, the focus – both digitally and pedagogically – should be on making it possible for learners to assemble the services into their environments themselves, in order to avail themselves of the support they need, when they need it, for the purposes they intend. The processes, methods, techniques, tools, and structures that students bring with them are at least as important as those created by their teachers. An integrated learning infrastructure needs to support these aspects at least as much as the interconnections between software tools. Again, it is necessary to think of the environment as considerably more than just a set of digital components that it uses but one that includes the people, the spaces they inhabit, and the things that they do.  Pedagogically as well as technically, there may be a need to support students in making the best use of all of that, for instance to search well, to find people that can help them to learn, to organize their own learning process, but such support Is, again, a service on which students may draw, not a teacher-determined requirement. And those pedagogies themselves need to adapt: for example, those that rely on rewards and punishments to enforce compliance must be excised, while those that provide learners with autonomy should be amplified. New pedagogies will be needed that acknowledge the many teachers in a learner’s environment, that help them to traverse the complexity of it, to leverage the advantages and to avoid the pitfalls. Teachers will need to let go, but stay close.

Tools that involve engagement with others – the means to share, the means to discuss, the means to work together, schedule meetings, and so on – are connection points in learners’ environments that cannot usually be completely controlled by any one of them, because of the need to at least agree protocols through which to engage and, in many cases, the systems which they will use to interact . One way to deal with this problem is to make a decision to use a  small range of tools, ideally in consultation with students. A better approach is to use tools that give students a choice of toolset, using protocols or standards such as SMTP, Jabber,  iCal, WebMention, ActivityStreams, or NNTP. However, few new standards have gained traction in recent years thanks to the dominance of closed social media monoliths intent on locking users in to their systems, so this may unnecessarily limit the range of systems that may be used. Another approach, commonly used in Connectivist approaches to learning, is to aggregate what learners provide themselves, using standards like RSS or Atom, or proprietary APIs offered by tool providers, or mailing systems to collect what students have shared elsewhere. If that is impossible, even simple copy-and-paste by human beings (students, teachers, or others) may be sufficient to connect multiple systems: not everything in an ILI needs to be implemented in software. For example, student blogs may be shared through flexible  technologies such as email and messaging apps, then copied by themselves or by their teachers into shared wikis. One  interesting benefit of such approaches is that they can support both diversity and manageability, inasmuch as the management burden may be shared by the participants rather than taken on by a single teacher or institution. Students may choose which tools they use, rather than having them chosen by the teacher. This is the principle used by Connectivist MOOCs (Downes, 2008), in which one site aggregates the shared artefacts created in many different learner-managed systems.

A learner’s environment consists of much more than the digital tools and systems offered by an institution. While, to a large extent, much of this environment may be unknowable to their designated teachers, there is much value to those who seek to support student learning in discovering how they are learning, and what constitutes their learning environment. Learning – the process, the tools, and the ways of learning, and not just the products – must be made visible if teachers, including other students, are to help learners to learn (Hattie, 2013). Much use can be made of pedagogical approaches such as shared learning diaries or blogging, and some careful use may even be made of automated systems that indicate presence, or that record traces of visits, as long as their role is to provide support for understanding student learning, and not to provide the teacher with means to control of the student. Beyond individual courses, there may be much pedagogical value in encouraging learners to share their learning experience through media such as blogs, microblogs, and other online tools, which may (as long as means are available for the student to control their privacy as needed) be aggregated and shared across their whole distributed, diffuse environment. Rather than replicating the necessarily closed and time-limited nature of the classroom, the artefacts of learning and the relationships that are developed in the process may persist indefinitely. Connectivist MOOCs provide a useful model for this. For example, Cormier (2014) talks of ‘Zombie MOOCs’ in which learning and interaction persist long after the course itself is over.

Bringing about such changes at an institutional level requires both bottom-up and top-down support. Teacher’s pedagogies are normally  more malleable than digital tools, because they can adapt rapidly to any tools: they are, in Brand’s terms (Brand, 1997), the ‘stuff’. However, they are therefore also the most constrained by the structures into which they must slot, and the least able to significantly impact things at structural level.  A single teacher, or even a small group of teachers pressing for change is therefore unlikely to sway either institutional policy or the design of the LMS because, as we have seen, one LMS must address the needs of all, so anything that changes it must suit everyone.

From the top down, replacing the LMS with an integrated learning infrastructure is a necessary step towards breaking out of the vicious loops that prevent the pedagogies and structures from evolving. At first, an ILI will naturally resemble the LMS it replaces, because its boundaries will continue to be largely determined by the less flexible layers above it: the institutional forms and structures such as courses, credentials, legislation, and teachers’ employment contracts. It is important to remember that the LMS was originally designed not just to replicate classroom behaviours but to fit into the larger, slower-changing structures and systems of institutions, and that significant changes in how we teach will not occur unless those structures and systems also evolve. They create the boundaries within which the ILI operates and, to a large extent, are not just containers of it, but part of it. An infrastructure is not just the digital tools but also the human-enacted methods, rules, protocols, and standards that accompany it. It is not just what we use, but the ways that we use it.  However, unlike the LMS, in an ILI those boundaries will be malleable. This opens up opportunities for the structure, skin, and services to in turn change.

The opportunities for change may not be taken, at first, at least in part because the signals (such as qualified students, their credentials, and so on) that pass in and out of the boundaries of the university will go to and from governments, employers, and other institutions that may not be prepared for radical change, even if the institution itself is committed to it. If, say, other institutions insist on grade point averages for standardized courses, then it will be difficult to completely avoid providing them, or something that is recognizably equivalent.  However, the adjacent possible empty niches (Kauffman, 2019) that an ILI supports will inevitably be filled by those who see the opportunities it entails, from courses whose lengths are pedagogically determined, to integration of lifelong and workplace learning, to new forms of credentials and learning. Perhaps, if enough institutions start to adopt such practices, we may break free of the insular single-institution model of education altogether. Out of this may grow a truly learner- and learning-driven future, in which learners draw on services from multiple educational providers, leading to a vast participative system in which institutions meld or blend to offer support for learning not just any time and any place, but every time and every place.

References

Blum, S. D., & Kohn, A. (2020). Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press.

Bouygues, H. L. (2019). Does Educational Technology Help Students Learn? https://reboot-foundation.org/does-educational-technology-help-students-learn/

Brand, S. (1997). How buildings learn. Phoenix Illustrated.

Brown, M., Dehoney, J., & Millichap, N. (2015). The next generation digital learning environment. A Report on Research. ELI Paper. Louisville, CO: Educause April, 5(1), 1-13.

Cormier, D. (2014). Community learning – the zombie resurrection. http://davecormier.com/edblog/2014/05/25/community-learning-the-zombie-resurrection/

Downes, S. (2008). Places to Go: Connectivism & Connective Knowledge. Innovate, 5(1). http://www.innovateonline.info/pdf/vol5_issue1/Places_to_Go-__Connectivism_&_Connective_Knowledge.pdf

Dron, J.  (2006). Any color you like, as long as it’s Blackboard®.  In. Hawaii: AACE.

Dron, J., & Anderson, T. (2014). Teaching crowds: Learning & Social Media. AU Press. http://teachingcrowds.ca

Dron, J. (2016). P-learning’s unwelcome legacy. TD Tecnologie Didattiche, 24(1), 72-81. http://www.tdjournal.itd.cnr.it/article/view/891

Dron, J. (2021). Educational technology: what it is and how it works. AI & SOCIETY, 1-12. https://doi.org/10.1007/s00146-021-01195-z

Greene, E. B. (1928). The relative effectiveness of lecture and individual reading as methods of college teaching. Genetic Psychology Monographs.

Hattie, J. (2013). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Taylor & Francis.

Kauffman, S. A. (2019). A World Beyond Physics: The Emergence and Evolution of Life. Oxford University Press.

Kelly, K. (2009). Triumph of the Default. http://www.kk.org/thetechnium/archives/2009/06/triumph_of_the.php

Keuning, H., Jeuring, J., & Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE), 19(1), 1-43.

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. http://www.alfiekohn.org/blogs/personalized/

Lane, D. C., & Good, C.  (2019). OERu’s delivery model for changing times: An Open Source NGDLE.  In World Conference on Online Learning, Dublin, Ireland. https://docs. oeru. org/s/fXQk2rJbzWCk8ia.

Martin, F., Chen, Y., Moore, R. L., & Westine, C. D. (2020). Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 2018. Educational Technology Research and Development, 68(4), 1903-1929.

Martindale, T., & Dowdy, M. (2010). Personal learning environments. Emerging technologies in distance education, 177, 193.

O’Neill, R.V., DeAngelis, D.L, Waide, J. B., & Allen, T. F. H. (1986). A Hierarchical Concept of Ecosystems. Princeton University Press.

Postman, N. (1998). Five things we need to know about technological change. Denver, Colorado, 28.  https://student.cs.uwaterloo.ca/~cs492/papers/neil-postman–five-things.html

Protopsaltis, S., & Baum, S. (2019). Does Online Education Live Up to its Promise? A Look at the Evidence and Implications for Federal Policy.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications.

Tichavsky, L. P., Hunt, A. N., Driscoll, A., & Jicha, K. (2015). “It’s Just Nice Having a Real Teacher”: Student Perceptions of Online versus Face-to-Face Instruction. International Journal for the Scholarship of Teaching and Learning, 9:2. http://digitalcommons.georgiasouthern.edu/ij-sotl/vol9/iss2/2/

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500-1509.

Wilson, S. (2008). Patterns of personal learning environments. Interactive learning environments, 16(1), 17-34.

Wilson, S., Sharples, P., & Griffiths, D.  (2008). Distributing education services to personal and institutional systems using Widgets.  In Proc. Mash-Up Personal Learning Environments-1st Workshop MUPPLE 8 (pp. 25-33).

Yen, C.-J., Tu, C.-H., Sujo-Montes, L. E., Harati, H., & Rodas, C. R. (2019). Using personal learning environment (PLE) management to support digital lifelong learning. International Journal of Online Pedagogy and Course Design (IJOPCD), 9(3), 13-31.