Ernst & Young fined $100 million after employees cheated in exams

Not just any exams: ethics exams.

These are the very accountants who are supposed to catch cheats. I guess at least they’ll understand their clientele pretty well.

But how did this happen? There are clues in the article:

“Many of the employees interviewed during the federal investigation said they knew cheating was a violation of the company’s code of conduct but did it anyway because of work commitments or the fact that they couldn’t pass training exams after multiple tries.” (my emphasis).

I think there might have been a clue about their understanding of ethical behaviour in that fact alone, don’t you? But I don’t think it’s really their fault: at least, it’s completely predictable to anyone with even the slightest knowledge of how motivation works.

If passing the exam is, by design, much more important than actually being able to do what is being examined, then of course people will cheat. For those with too much else to do or too little interest to succeed, when the pressure is high and the stakes are higher, it’s a perfectly logical course of action. But, even for all the rest who don’t cheat, the main focus for them will be on passing the exam, not on gaining any genuine competence or interest in the subject. It’s not their fault: that’s how it is designed. In fact, the strong extrinsic motivation it embodies is pretty much guaranteed to (at best) persistently numb their intrinsic interest in ethics, if it doesn’t extinguish it altogether. Most will do enough to pass and no more, taking shortcuts wherever possible, and there’s a good chance they will forget most of it as soon as they have done so.

Just to put the cherry on the pie, and not unexpectedly, EY refer to the process by which their accountants are expected to learn about ethics as ‘training’ and it is mandatory. So you have a bunch of unwilling people who are already working like demons to meet company demands, to whom you are doing something normally reserved for dogs or AI models, and then you are forcing them to take high-stakes exams about it, on which their futures depend. It’s a perfect shit storm. I’d not trust a single one of their graduates, exam cheats or not, and the tragedy is that the people who were trying to force them to behave ethically were the ones directly responsible for their unethical behaviour.

There may be a lesson or two to be learned from this for academics, who tend to be the biggest exam fetishists around, and who seem to love to control what their students do.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/14163409/ernst-young-fined-100-million-after-employees-cheated-in-exams

Solving The Wrong Problems: Why Online Education Is and Must Be Different from In-Person Education – slides from my invited talk at ICEMI 2022

icemi22

These are the slides from my invited talk at the 11th International Conference on Education and Management Innovation (ICEMI 2022), June 11th. The talk went down well – at least, I was invited to repeat the performance at a workshop (where I gave a very similar presentation today – if you’ve seen one, you probably know the content of the other!) and to give a keynote later in the year.

It’s about how methods of teaching that solve problems for in-person teachers don’t apply online, and it provides a bit of advice on online-native approaches. I’ve talked quite a bit about this over the past decade so there’s not much new in it apart from minor refinements, though I have put a greater emphasis on what goes on outside the classroom in physical institutions because I’m increasingly thinking that this matters way more than we normally acknowledge. Notably, I discuss the ways that physical institutional structures and regulations provide significant teaching functions of their own, meaning that in-person teachers can be absolute rubbish or (in some subject areas or topics) even fail to turn up, and students can still learn pretty well. This helps to explain the bizarre phenomenon that, across much of in-person academia, professors and lecturers are not expected to learn how to teach (and many never do).

Here’s the abstract…

In-person educational institutions teach, at least as much as the individual teachers they employ. Students are taken out of their own environments and into that of the institution, signalling intent to learn. The physical environment is built for pervasive learning, from common rooms, to corridors, to campus cafes; students see one another learning, share learning conversations, learn from one another. Even the act of walking from classroom to classroom makes events within them more salient. Structures such as courses, timetables, semesters, and classes solve problems of teaching efficiently within the constraints of time and space but impose great constraints on how teaching occurs, and create multiple new pedagogical and management problems of their own. The institution’s regulations, expectations, and norms play a strong pedagogical role in determining how, and when learning occurs. Combined with other entrenched systems and tools like credentials, textbooks, libraries, and curricula, a great deal of the teaching process occurs regardless of teachers. What we most readily recognize as ‘good’ teaching overcomes the problems caused by these in-person environments, and exploits their affordances.

Online institutions have radically different problems to solve, and radically different affordances to exploit, so it makes no sense to teach or manage the learning process in the same ways. Online, students do not inhabit the environment of the institution: the institution inhabits the environment of the student. It is just one small part of the student’s physical and virtual space, shared with billions of other potential teachers (formal or not) who are a click, a touch, or a glance away. The institution is just a service, not the environment in which learning occurs. The student picks the time, the space, the pace, and virtually all the surrounding supports of the learning process. Teachers cannot actively control any of this, except through the use of rewards, punishments, and the promise of credentials, that force compliance but that are antagonistic to effective or meaningful learning. In this talk, I will discuss the implications of this inverted dynamic for pedagogy, motivation, digital system design, and organizational structures & systems for online learning.

 

 

 

English version of my 2021 paper, “Technology, technique, and culture in educational systems: breaking the iron triangle”

Technology, technique, and culture in educational systems: breaking the iron triangle

This is the (near enough final) English version of my journal paper, translated into Chinese by Junhong Xiao and published last year (with a CC licence) in Distance Education in China. (Reference: Dron, Jon (2021).  Technology, technique, and culture in educational systems: breaking the iron triangle (translated by Junhong Xiao). Distance Education in China, 1, 37-49. DOI:10.13541/j.cnki.chinade.2021.01.005).

The underlying theory is the same as that in my paper Educational technology: what it is and how it works (Reference: Dron, J. Educational technology: what it is and how it works. AI & Soc 37, 155–166 (2022). https://doi.org/10.1007/s00146-021-01195-z direct link for reading, link to downloadable preprint) but this one focuses more on what it means for ways we go about distance learning. It’s essentially about ways to solve problems that we created for ourselves by solving problems in the context of in-person learning that we inappropriately transferred to a distance context.

Here’s the abstract:
This paper presents arguments for a different way of thinking about how distance education should be designed. The paper begins by explaining education as a technological process, in which we are not just users of technologies for learning but coparticipants in their instantiation and design, implying that education is a fundamentally distributed technology. However, technological and physical constraints have led to processes (including pedagogies) and path dependencies in In-person education that have tended to massively over-emphasize the designated teacher as the primary controller of the process. This has resulted in the development of many counter technologies to address the problems this causes, from classrooms to grades to timetables, most of which have unnecessarily been inherited by distance education. By examining the different strengths and weaknesses of distance education, the paper suggests an alternative model of distance education that is more personal, more situated in communities and cultures, and more appropriate to the needs of learners and society.

I started working on a revised version of this (with a snappier title) to submit to an English language journal last year but got waylaid. If anyone is interested in publishing this, I’m open to submitting it!

Higher Education: an Owner's Guide – slides from my Times Higher Ed Student Festival keynote

Higher Education: an Owner’s Guide, slides from my keynote at the THE Student Festival, UK, 2021

With a possible audience of thousands, and without a clue they were there because the Zoom output was streamed to a different site (weird), I talked very fast about my experience of higher education for about 20 minutes at the THE UK Student Festival yesterday. The talk was recorded and will be used again for the THE Canada Student Festival later in the month. It’s a huge event – over 8000 enrolled (though not all attending every session) – with quite a lot of other keynotes and a great many other talks, panels, and discussions, aimed at helping students starting out in higher education. The audience was very different from those I normally talk to, and the  (very sensible) strict 20-minute format gave me a lot less leeway than the usual hour allowed, so I found it interestingly challenging. These are the slides I used.

The brief I was given was not to preach, but to share my experience of higher education, as a student and as an educator. My personal agenda was to talk about why and how online learning is worth doing (especially at AU), so I tried (a bit clunkily) to aim my story in that direction. I failed to remember to mention some key things, I spent too long on others, and I suspect that the most memorable message that came through was, for in-person students living on campus, to get a kettle (it’s a great way to make lots of new friends fast) but, hopefully, my bigger message got through to some. kettle

The essential point of my wild ramble was not that kettles are the solution to success in higher education, but that students and the rest of us should take ownership of our own education: it should be done by us, not to us. We should learn the way we want to learn, and we should learn what we want to learn; we should seek adventure and challenge rather than easy pickings; and we should hang out with interesting people and/or those we care about, because that’s how most learning happens, as well as being a large part of what makes it meaningful. I noted that we should focus on learning and should to try to ignore grades as much as possible, because grades destroy the love of doing something simply because we enjoy it. Essentially, my advice was about finding the things that intrinsically motivate us and reducing the effects of things that demotivate us. My own educational journey, and (I think) that of most committed educators, has largely followed that path: that’s how we thrived in a system not conducive to intrinsic motivation, and it’s the path we try to encourage our own students to take. I observed that it is much easier to own your own education when it is done online, at least if it is done in ways that take advantage of the medium, and not through a pale simulacrum of in-person teaching, because the teacher cannot be in control, the level of challenge is much more controllable, and there are way more people (online and in your own environment) who can support you.

Challenges of the Physical: slides from my keynote at XII Conferência Internacional de Tecnologias de Informação e Comunicação na Educação, September 2021

Here are the slides from my opening keynote today for the XII Conferência Internacional de Tecnologias de Informação e Comunicação na Educação in Portugal. first slide of the presentation

The conference theme was ‘challenges of the digital’ so I thought it might be fun to reverse the problem, and to think instead about the challenges of in-person education. In this presentation I imagined a world in which in-person teaching had never been invented, and presented a case for doing so. In fairness, it was not a very good case! But I did have fun using some of the more exotic voice changing features of my Voicelive Play vocal processor (which I normally use for performing music), presenting some of the arguments against my suggestions in different voices using a much better mic than my usual (pretty good) Blue Yeti. I might not use the special effects again that often, but I was quite impressed with the difference the better microphone made.

My central points (mostly implicit until the end) were:

  • That the biggest challenge of the digital is all the baggage that we have inherited from in-person teaching, and our continuing need to interoperate with in-person institutions.
  • That pedagogies are neither universal nor neutral. They are solutions to problems of learning in a particular context, in assembly with countless constraints and possibilities provided by that context: people, tools, structures, methods, systems, and so on.
  • That solutions to learning in a physical context – at least in the one-to-many model of traditional education systems – inevitably lead to a very strong power imbalance between teacher and learner, where the teacher is in control of every moment that the teaching event occurs. This has many repercussions, not least of which being that needs for autonomy and competence support are very poorly addressed (though relatedness comes for free), so it is really bad for intrinsic motivation.
  • Thus, the pedagogies of physical spaces have to compensate for the loss of control and achievable challenge that they naturally entail.
  • That the most common approach – and, again, an almost inevitable (i.e. the shortest path) follow-on from teaching a lot of people at once – involves rewards and punishments, that massively impair or destroy intrinsic motivation to learn and, in most cases, actively militate against effective learning.
  • That the affordances of teaching everyone the same thing at once lead fairly naturally to credentials for having learned it, often achieved in ‘efficient’ ways like proctored exams that are incredibly bad for learning, and that greatly reinforce the extrinsic motivation that is already highly problematic in the in-person modality. The credentials, not the learning, become the primary focus.
  • That support for autonomy and competence are naturally high in online learning, though support for relatedness is a mix of good and bad. There is no need for teachers being in control and, lacking most of the means of control available to in-person teachers, the only reliable way to regain it is through rewards and punishments which, as previously mentioned, are fatal to intrinsic motivation.
  • That the almost ubiquitous ways that distance educators inherit and use the pedagogies, methods, and structures of in-person learning – especially in the use of coercion through rewards and punishments (grades, credentials, etc) but also in schedules, fixed-length courses, inflexible learning outcomes, etc – are almost exactly the opposite of what its technologies can best support.

Towards the end, acknowledging that it is difficult to change such complex and deeply entangled systems (much though it is to be desired) I presented some ways of reducing the challenges of the physical in online teaching, and regaining that lost intrinsic motivation, that I summarized thus:

  • Let go (you cannot and should not control learning unless asked to do so), but stay close;
  • Make learning (not just its products) visible (and, in the process, better understand your teaching);
  • Make learning shared (cooperation and, where possible, collaboration built in from the ground up);
  • Don’t ever coerce (especially not through grades);
  • Care (for learners, for learning, for the subject).

It’s a theme that I have spoken and written of many, many times, but (apart from the last few slides) the way I presented it this time was new for me. I had fun pretending to be different people, and the audience seemed to like it, in a challenging kind of a way. There were some great questions at the end, not all of which I had time to answer, though I’m happy to continue the conversation here, or via Twitter.

Mediaeval Teaching in the Digital Age (slides from my keynote at Oxford Brookes University, May 26, 2021)

 front slide, mediaeval teaching

These are the slides from my keynote today at the Oxford Brookes “Theorizing the Virtual” School of Education Research Conference. As theorizing the virtual is pretty much my thing, I was keen to be a part of this! It was an ungodly hour of the day for me (2am kickoff) but it was worth staying up for. It was a great bunch of attendees who really got into the spirit of the thing and kept me wide awake. I wish I could hang around for the rest of it but, on the bright side, at least I’m up at the right time to see the Super Flower Blood Moon (though it’s looking cloudy, darn it).  In this talk I dwelt on a few of the notable differences between online and in-person teaching. This is the abstract…

Pedagogical methods (ways of teaching) are solutions to problems of helping people to learn, in a context filled with economic, physical, temporal, legal, moral, social, political, technological, and organizational constraints. In mediaeval times books were rare and unaffordable, and experts’ time was precious and limited, so lectures were a pragmatic solution, but they in turn created more problems. Counter-technologies such as classes, classrooms, behavioural rules and norms, courses, terms, curricula, timetables and assignment deadlines were were devised to solve those problems, then methods of teaching (pedagogies) were in turn invented to solve problems these counter-technologies caused, notably including:
· people who might not want (or be able) to be there at that time,
· people who were bored and
· people who were confused.
Better pedagogies supported learner needs for autonomy and competence, or helped learners find relevance to their own goals, values, and interests. They exploited physical closeness for support, role-modelling, inspiration, belongingness and so on. However, increasingly many relied on extrinsic motivators, like classroom discipline, grades and credentials to coerce students to learn. Extrinsic motivation achieves compliance, but it makes the reward or avoidance of the punishment the goal, persistently and often permanently crowding out intrinsic motivation. Intelligent students respond with instrumental approaches, satisficing, or cheating. Learning seldom persists; love of the subject is subdued; learners learn to learn in ineffective ways. More layers of counter-technologies are needed to limit the damage, and so it goes on.
Online, the constraints are very different, and its native forms are the motivational inverse of in-person learning. An online teacher cannot control every moment of a learner’s time, and learners can use the freedoms they gain to take the time they need, when they need it, to learn and to reflect, without the constraints of scheduled classroom hours and deadlines. However, more effort is usually needed to support their needs for relatedness. Unfortunately, many online teachers try (or are required) to re-establish the control they had in the classroom through grading or the promise of credentials, recreating the mediaeval problems that would otherwise not exist, using tools like learning management systems that were designed (poorly) to replicate in-person teaching functions. These are solutions to the problems caused by counter-technologies, not to problems of learning.
There are better ways, and that’s what this session is about.

front slide, mediaeval teaching

Educational technology: what it is and how it works

https://rdcu.be/ch1tl

This is a link to my latest paper in the journal AI & Society. You can read it in a web browser from there, but it is not directly downloadable. A preprint of the submitted version (some small differences and uncorrected errors here and there, notably in citations) can be downloaded from https://auspace.athabascau.ca/handle/2149/3653. The published version should be downloadable for free by Researchgate members.

This is a long paper (about 10,000 words), that summarizes some of the central elements of the theoretical model of learning, teaching and technology developed in my recently submitted book (still awaiting review) and that gives a few examples of its application. For instance, it explains:

  • why, on average researchers find no significant difference between learning with and without tech.
  • why learning styles theories are a) inherently unprovable, b) not important even if they were, and c) a really bad idea in any case.
  • why bad teaching sometimes works (and, conversely, why good teaching sometimes fails)
  • why replication studies cannot be done for most educational interventions (and, for the small subset that are susceptible to reductive study, all you can prove is that your technology works as intended, not whether it does anything useful).

Abstract

This theoretical paper elucidates the nature of educational technology and, in the process, sheds light on a number of phenomena in educational systems, from the no-significant-difference phenomenon to the singular lack of replication in studies of educational technologies.  Its central thesis is that we are not just users of technologies but coparticipants in them. Our participant roles may range from pressing power switches to designing digital learning systems to performing calculations in our heads. Some technologies may demand our participation only in order to enact fixed, predesigned orchestrations correctly. Other technologies leave gaps that we can or must fill with novel orchestrations, that we may perform more or less well. Most are a mix of the two, and the mix varies according to context, participant, and use. This participative orchestration is highly distributed: in educational systems, coparticipants include the learner, the teacher, and many others, from textbook authors to LMS programmers, as well as the tools and methods they use and create.  From this perspective,  all learners and teachers are educational technologists. The technologies of education are seen to be deeply, fundamentally, and irreducibly human, complex, situated and social in their constitution, their form, and their purpose, and as ungeneralizable in their effects as the choice of paintbrush is to the production of great art.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/8692242/my-latest-paper-educational-technology-what-it-is-and-how-it-works

How distance changes everything: slides from my keynote at the University of Ottawa

These are the slides from my keynote at the University of Ottawa’s “Scaffolding a Transformative Transition to Distance and Online Learning” symposium today. In the presentation I discussed why distance learning really is different from in-person learning, focusing primarily on the fact that they are the motivational inverse of one another. In-person teaching methods evolved in response to the particular constraints and boundaries imposed by physics, and consist of many inventions – pedagogical and otherwise – that are counter-technologies designed to cope with the consequences of teaching in a classroom, a lot of which are not altogether wise. Many of those constraints do not exist online, and yet we continue to do very similar things, especially those that control and dictate what students should do, as well as when, and how they should do it. This makes no sense, and is actually antagonistic to the natural flow of online learning. I provided a few simple ideas and prompts for thinking about how to go more with the flow.

The presentation was only 20 minutes of a lively and inspiring hour-long session, which was fantastic fun and provided me with many interesting questions and a chance to expand further on the ideas.

uottawa2020HowDistanceChangesEverything

Joyful assessment: beyond high-stakes testing

Here are my slides from my presentation at the Innovate Learning Summit yesterday. It’s not world-shattering stuff – just a brutal attack on proctored, unseen written exams (PUWEs, pronounced ‘pooies’), followed by a description of the rationale, process, benefits, and unwanted consequences behind the particular portfolio-based approach to assessment employed in most of my teaching. It includes a set of constraints that I think are important to consider in any assessment process, grouped into pedagogical, motivational, and housekeeping (mainly relating to credentials) clusters. I list 13 benefits of my approach relating to each of those clusters, which I think make a pretty resounding case for using it instead of traditional assignments and tests. However, I also discuss outstanding issues, most of which relate to the external context and expectations of students or the institution, but a couple of which are fairly fundamental flaws (notably the extreme importance of prompt, caring, helpful instructor/tutor engagement in making it all work, which can be highly problematic when it doesn’t happen) that I am still struggling with.

Evaluating assessment

Exam A group of us at AU have begun discussions about how we might transform our assessment practices, in the light of the far-reaching AU Imagine plan and principles. This is a rare and exciting opportunity to bring about radical and positive change in how learning happens at the institution. Hard technologies influence soft more than vice versa, and assessments (particularly when tied to credentials) tend to be among the hardest of all technologies in any pedagogical intervention. They are therefore a powerful lever for change. Equally, and for the same reasons, they are too often the large, slow, structural elements that infest systems to stunt progress and innovation.

Almost all learning must involve assessment, whether it be of one’s own learning, or provided by other people or machines. Even babies constantly assess their own learning. Reflection is assessment. It is completely natural and it only gets weird when we treat it as a summative judgment, especially when we add grades or credentials to the process, thus normally changing the purpose of learning from achieving competence to achieving a reward. At best it distorts learning, making it seem like a chore rather than a delight, at worst it destroys it, even (and perhaps especially) when learners successfully comply with the demands of assessors and get a good grade. Unfortunately, that’s how most educational systems are structured, so the big challenge to all teachers must be to eliminate or at least to massively reduce this deeply pernicious effect. A large number of the pedagogies that we most value are designed to solve problems that are directly caused by credentials. These pedagogies include assessment practices themselves.

With that in mind, before the group’s first meeting I compiled a list of some of the main principles that I adhere to when designing assessments, most of which are designed to reduce or eliminate the structural failings of educational systems. The meeting caused me to reflect a bit more. This is the result:

Principles applying to all assessments

  • The primary purpose of assessment is to help the learner to improve their learning. All assessment should be formative.
  • Assessment without feedback (teacher, peer, machine, self) is judgement, not assessment, pointless.
  • Ideally, feedback should be direct and immediate or, at least, as prompt as possible.
  • Feedback should only ever relate to what has been done, never the doer.
  • No criticism should ever be made without also at least outlining steps that might be taken to improve on it.
  • Grades (with some very rare minor exceptions where the grade is intrinsic to the activity, such as some gaming scenarios or, arguably, objective single-answer quizzes with T/F answers) are not feedback.
  • Assessment should never ever be used to reward or punish particular prior learning behaviours (e.g. use of exams to encourage revision, grades as goals, marks for participation, etc) .
  • Students should be able to choose how, when and on what they are assessed.
  • Where possible, students should participate in the assessment of themselves and others.
  • Assessment should help the teacher to understand the needs, interests, skills, and gaps in knowledge of their students, and should be used to help to improve teaching.
  • Assessment is a way to show learners that we care about their learning.

Specific principles for summative assessments

A secondary (and always secondary) purpose of assessment is to provide evidence for credentials. This is normally described as summative assessment, implying that it assesses a state of accomplishment when learning has ended. That is a completely ridiculous idea. Learning doesn’t end. Human learning is not in any meaningful way like programming a computer or storing stuff in a database. Knowledge and skills are active, ever-transforming, forever actively renewed, reframed, modified, and extended. They are things we do, not things we have.

With that in mind, here are my principles for assessment for credentials (none of which supersede or override any of the above core principles for assessment, which always apply):

  • There should be no assessment task that is not in itself a positive learning activity. Anything else is at best inefficient, at worst punitive/extrinsically rewarding.
  • Assessment for credentials must be fairly applied to all students.
  • Credentials should never be based on comparisons between students (norm-referenced assessment is always, unequivocally, and unredeemably wrong).
  • The criteria for achieving a credential should be clear to the learner and other interested parties (such as employers or other institutions), ideally before it happens, though this should not forestall the achievement and consideration of other valuable outcomes.
  • There is no such thing as failure, only unfinished learning. Credentials should only celebrate success, not punish current inability to succeed.
  • Students should be able to choose when they are ready to be assessed, and should be able to keep trying until they succeed.
  • Credentials should be based on evidence of competence and nothing else.
  • It should be impossible to compromise an assessment by revealing either the assessment or solutions to it.
  • There should be at least two ways to demonstrate competence, ideally more. Students should only have to prove it once (though may do so in many ways and many times, if they wish).
  • More than one person should be involved in judging competence (at least as an option, and/or on a regularly taken sample).
  • Students should have at least some say in how, when, and where they are assessed.
  • Where possible (accepting potential issues with professional accreditation, credit transfer, etc) they should have some say over the competencies that are assessed, in weighting and/or outcome.
  • Grades and marks should be avoided except where mandated elsewhere. Even then, all passes should be treated as an ‘A’ because students should be able to keep trying until they excel.
  • Great success may sometimes be worthy of an award – e.g. a distinction – but such an award should never be treated as a reward.
  • Assessment for credentials should demonstrate the ability to apply learning in an authentic context. There may be many such contexts.
  • Ideally, assessment for credentials should be decoupled from the main teaching process, because of risks of bias, the potential issues of teaching to the test (regardless of individual needs, interests and capabilities) and the dangers to motivation of the assessment crowding out the learning. However, these risks are much lower if all the above principles are taken on board.

I have most likely missed a few important issues, and there is a bit of redundancy in all this, but this is a work in progress. I think it covers the main points.

Further random reflections

There are some overriding principles and implied specifics in all of this. For instance, respect for diversity, accessibility, respect for individuals, and recognition of student control all fall out of or underpin these principles. It implies that we should recognize success, even when it is not the success we expected, so outcome harvesting makes far more sense than measurement of planned outcomes. It implies that failure should only ever be seen as unfinished learning, not as a summative judgment of terminal competence, so appreciative inquiry is far better than negative critique. It implies flexibility in all aspects of the activity. It implies, above and beyond any other purpose, that the focus should always be on learning. If assessment for credentials adversely affects learning then it should be changed at once.

In terms of implementation, while objective quizzes and their cousins can play a useful formative role in helping students to self-assess and to build confidence, machines (whether implemented by computers or rule-following humans) should normally be kept out of credentialling. There’s a place for AI but only when it augments and informs human intelligence, never when it behaves autonomously. Written exams and their ilk should be avoided, unless they conform to or do not conflict with all the above principles: I have found very few examples like this in the real world, though some practical demonstrations of competence in an authentic setting (e.g. lab work and reporting) and some reflective exercises on prior work can be effective.

A portfolio of evidence, including a reflective commentary, is usually going to be the backbone of any fair, humane, effective assessment: something that lets students highlight successes (whether planned or not), that helps them to consolidate what they have learned, and that is flexible enough to demonstrate competence shown in any number of ways. Outputs or observations of authentic activities are going to be important contributors to that. My personal preference in summative assessments is to only use the intended (including student-generated) and/or harvested outcomes for judging success, not for mandated assignments. This gives flexibility, it works for every subject, and it provides unquivocal and precise evidence of success. It’s also often good to talk with students, perhaps formally (e.g. a presentation or oral exam), in order to tease out what they really know and to give instant feedback. It is worth noting that, unlike written exams and their ilk, such methods are actually fun for all concerned, albeit that the pleasure comes from solving problems and overcoming challenges, so it is seldom easy.

Interestingly, there are occasions in traditional academia where these principles are, for the most part, already widely applied. A typical doctoral thesis/dissertation, for example, is often quite close to it (especially in more modern professional forms that put more emphasis on recording the process), as are some student projects. We know that such things are a really good idea, and lead to far richer, more persistent, more fulfilling learning for everyone. We do not do them ubiquitously for reasons of cost and time. It does take a long time to assess something like this well, and it can take more time during the rest of the teaching process thanks to the personalization (real personalization, not the teacher-imposed form popularized by learning analytics aficionados) and extra care that it implies. It is an efficient use of our time, though, because of its active contribution to learning, unlike a great many traditional assessment methods like teacher-set assignments (minimal contribution) and exams (negative contribution).  A lot of the reason for our reticence, though, is the typical university’s schedule and class timetabling, which makes everything pile on at once in an intolerable avalanche of submissions. If we really take autonomy and flexibility on board, it doesn’t have to be that way. If students submit work when it is ready to be submitted, if they are not all working in lock-step, and if it is a work of love rather than compliance, then assessment is often a positively pleasurable task and is naturally staggered. Yes, it probably costs a bit more time in the end (though there are plenty of ways to mitigate that, from peer groups to pedagogical design) but every part of it is dedicated to learning, and the results are much better for everyone.

Some useful further reading

This is a fairly random selection of sources that relate to the principles above in one way or another. I have definitely missed a lot. Sorry for any missing URLs or paywalled articles: you may be able to find downloadable online versions somewhere.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. Retrieved from https://www.jhsph.edu/departments/population-family-and-reproductive-health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-learning.pdf

Boud, D. (2007). Reframing assessment as if learning were important. Retrieved from https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in organizational change and development, 1, 129-169.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3/4), 325-346.

Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3(3), 220-233.

Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes (Kindle ed.). Mariner Books. (this one is worth forking out money for).

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33.

Kohn, A. (2015). Four Reasons to Worry About “Personalized Learning”. Retrieved from http://www.alfiekohn.org/blogs/personalized/ (check out Alfie Kohn’s whole site for plentiful other papers and articles – consistently excellent).

Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-Determination research (pp. 183-203). Rochester, NY: The University of Rochester Press.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications. (may be worth paying for if such things interest you).

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Cairo: Ford Foundation. http://www.managingforimpact.org/sites/default/files/resource/outome_harvesting_brief_final_2012-05-2-1.pdf.