Is China really the educational powerhouse that the PISA rankings suggest? (tl;dr: not even close)

Administered by the OECD, PISA is basically a set of tests, adapted to each country, that attempt to measure educational performance across a range of skills in order to rank educational systems around the world. The rankings really matter to many countries, and help to determine educational policies across the planet, being especially impactful when countries don’t do well. Often, a low PISA score triggers educational reform (not always ending well), but sometimes countries just stop playing the game. India, for instance, dropped out a decade ago after coming second from the bottom, complaining of lack of adaptation to the Indian context (which is totally fair – India is incredibly diverse, so one measure absolutely does not fit all) though it will be back again next year. There are many reasons to dislike PISA, but the one I want to highlight here is Goodhart’s Law that, when a measure becomes a target, it ceases to be a good measure.

This article – a report on an interview with Andreas Schleicher, OECD Director of Education and Skills (a very smart fellow) – provides some useful food for thought. Though it focuses on China as a case in point, the interview is not so much about China’s ‘success’ as it is about PISA and its limitations in general. Among Schleicher’s more interesting insights is the fact that China’s test results came solely from its four most highly developed and economically successful provinces. These are very unrepresentative of the whole. In fact, China replaced Guangdong in its submission this time round because it was blamed for poorer performance last time, suggesting that the Chinese government’s involvement with PISA is far more concerned with appearing effective on the International stage – on presenting a facade – than on actually improving learning. PISA is a test for countries, and some are quite happy to cheat on the test.

In fact, the biggest contributing factor to test results is, of course and as always, economic. Schleicher notes that, worldwide, the top 10% socioeconomically advantaged students have for at least 10 years consistently outperformed the 10% most disadvantaged students in reading by 141 score points, which equates to approximately three year’s worth of schooling. It is not news that by far the most productive way to improve the effectiveness of educational systems would be to diminish wealth inequalities. It is, though, worth noting that schools play a relatively small role in Chinese education, especially among more prosperous families, with vast amounts of (paid, private) tuition occurring outside schools. Similar extracurricular tuition patterns occur in several of the other highest ranking PISA countries, such as South Korea, Singapore, Japan, Hong Kong, and Taiwan. It is significant that, in these countries, test scores are extremely important in almost every way – economically, culturally, socially, and more – so there is a lot of teaching focused on test results at the expense of almost everything else.

It is also notable – and almost certainly a direct consequence of tests’ importance – that over 80% of Chinese students admit to cheating, which might be more than a minor contributor to the good results. In fairness, cheating rates for the US and Canada are also not too far short of that, correctly implying a serious endemic malaise with our educational systems worldwide (Goodhart’s Law, again), so this is just a relatively slight difference of degree, not of kind. Given the large amount of time spent learning outside school, the high levels of cheating, and the cherry-picking of top performing provinces, the implications are that, far from having a world-leading education system, teaching in China is actually really awful, on average. Among the things that can be gleaned from PISA results are that China performs very badly on productivity (points per hour of learning), and ranks 8th from bottom on life satisfaction for students. It is essentially a failure, by any reasonable measure. The PISA ranking is not quite a fiction but it is close. At least in the case of the high overall placing of China, it certainly fails to correctly measure the effectiveness of the educational system, if results are taken at face value.

There appear to be two distinct patterns among those countries that consistently achieve high PISA results, that appear to divide along broadly cultural lines. The first group includes the likes of China, South Korea, Japan, and Taiwan (all quite notable examples of what Hofsteder describes as collectivist cultures), with high levels of out-of-school tuition, a strong educational emphasis on test scores, and great personal penalties for failure. These countries seem to achieve their high ranking by a very strong focus on passing the tests, with high penalties for failure and great significance for success. As a consequence, their educational systems cannot be seen as standalone causes but, rather, as creators of problems that have to be overcome by other means (most notably in the form of extra-curricular assistance that funds a booming personal tuition economy).  Standard bearers for the other main pattern are Finland and Estonia, as well as Switzerland, and Canada (though the latter two devolve educational responsibility to canton/province, so they are less consistently successful in the rankings). In Hofstede’s terms, these are more individualist societies. In this group, test scores (slightly) tend to be seen as a measure of only one of several consequences of teaching, rather than being the primary motivation for doing it. I am certainly culturally biased, but I cannot help but think this is a better way of going about the process: education is for society, much more than for the individual, and certainly not for economic gain, so it must be understood across many dimensions of value. Whether they agree with me or not, I am almost certain that most educators everywhere would like to think that education is about much more than achieving good test scores. It is only a matter of degree, though. Education in all countries I am aware of relies on extrinsic motivation, and there are large pockets of excellence in the first group and large pockets of awfulness in the second. Averages are a stupid way to evaluate a whole country’s educational system, and they conceal great diversity. The boundaries are also blurred. Estonia, for instance, that is singled out in the article as a success story due to its rapid rise through the rankings, actually also makes extensive use of extra tuition in the form of ‘long day groups’ that take place in schools after curricular instruction. Estonia is no worse than most other countries in this regard, and in some ways superior because such long day groups take the place of at least some of the homework that is widely required in many countries, despite a singular lack of evidence that (on average) it has more than a tiny effect on learning. At least Estonia’s approach involves a modicum of good education theory and evidence to support it.

Overall, I think the main thing that is revealed by the PISA process is that average test scores are, for the most part, an extremely poor means of comparing education systems. Given that it is useful for a government to know how their policies are working, there does need to be some way for them to observe how schools are doing, but it would seem more sensible to rely on trained inspectors reviewing schools, their teaching, the work of children, etc, than on test scores. At the very least they should be considering signs of happiness, motivation, community, and social achievement at least as much as academic achievement. However, Goodhart’s Law would cause its usual harm if such things became the dominant measures of success, and more than the lightest of inspections would normally cause more harm than good. I experienced something not too far removed from this (in the form of OFSTED inspections) in the UK as a parent and school governor back in the 1990s. The results were not pretty. For about a year leading up to them teachers’ workloads were massively strained by the need to report on everything, students suffered, resentments piled up, everyone suffered. Though OFSTED reports did sometimes lead to improvements in particularly bad schools, the effects on the vast majority of schools (and especially on teachers) were disastrous, often radically disrupting work, increasing stress levels beyond reasonable bounds, and leading to more than a few resignations and early retirements from the best, most dedicated teachers who could barely cope with the workloads at the best of times. They were forced to become bureaucrats, which is a role to which teachers tend to be very poorly suited. It was (and, I believe, may still be) beyond stupid, despite best intentions.

What is really needed is something more collegial, that is focused on improvement rather than judgment, that celebrates and builds on success rather than amplifying failure, where everyone involved in the process benefits and no one suffers. The whole point (as far as I understand it) is to improve what we do, not to blame those who fail. Appreciative Inquiry is a good start. Simple things like peer observation (with no penalties, no judgments, just formative commentary) can be more than adequate for the most part at a local level, and are beneficial to both observer and observed. Maybe – if someone thinks it necessary – inspectors (volunteers, perhaps, from the teaching profession) could look at samples of student work from further afield with a similarly positive, formative attitude. It might not provide numbers to compare but, if there were enough of a culture of sharing across the whole sector, and if inspectors came from across the geographical and cultural spectrum, it ought to be good enough to improve practice, and to spread good ideas around, so the intent would be achieved. Governments could receive reports on what actually matters – that things are getting better – rather than on what does not (that things are bad, according to some unreliable measurement that compares nothing of any real value to educators, students, or society). Teaching is a deeply soft technology that cannot be reductively simplified to a relationship of entailment. It can, though, as a lived, creative, social process, be improved. This should be the goal of all teachers, and of all those who can influence the process, including governments. PISA only achieves such results in a tiny minority of extreme cases. For the most part, it actively militates against them because it substitutes education – in all its rich complexity – for test scores. These are not even a passable proxy. They are a gross distortion, an abhomination that can trivially be turned to evil, self-serving purposes without in any way improving learning. Schleicher fully understands this. I wish that the people who his organization serves did too.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/5209267/is-china-really-the-educational-powerhouse-that-the-pisa-rankings-suggest-tldr-not-even-close

Excellent news: Twitter Makes A Bet On Protocols Over Platforms.

Well this is good news! Of course, the road to Hell is paved with good intentions and there is much that could go wrong in between plan and execution, but it seems that Twitter is recommitting itself to openness, standards, and the use of protocols for a federated social Web (see also https://twitter.com/biz/status/1204784388107636737 and https://twitter.com/jack/status/1204766078468911106 for the announcements by Twitter’s founders). It is a bit worrying that Twitter wants to help invent a new protocol when there are plenty of established ones that already exist (ActivityPub, OpenSocial, FOAF, XMPP, OStatus, OpenID, OAuth, PubSubHubbub, Zot, Diaspora, etc, etc). Also, there is already a pretty serviceable Twitter competitor in the form of Mastodon, that does most of what they seem to want to do. However, the fact that they are thinking about protocols rather than platforms at all is very heartening. The world needs much much more of this.

Twitter, as it evolved in its first couple of years, was brilliant. What made it great was that it could act as a highly efficient social bookmarking system *plus* commentary *plus* folksonomy, *plus* instant messaging, *plus* social networking, all through one incredibly simple, flexible, open field.  It was, in part, a descendant of social bookmarking systems that people like me developed in the 90s, but there were no predetermined fields for URLs (you could have more than one, or none at all); there were no predetermined categories; the tags (#hashtags) were trivially easy to include, without separate fields (this is what makes it highly supportive of social sets, in which the topic matters more than the person); and it had the lowest threshold social networking (especially through @mentions), again without the need for separate fields. It was a single small text box that did everything, and that could be used to share more or less anything with more or less anyone but, thanks to its size,  was primarily used to connect to other things. Part of what made this so cool is that #hashtags and @mentions were not designed into Twitter at the start, but emerged memetically from practice: the system evolved (at first) through a collective design process. Twitter’s implementation of such things in software ingeniously used automation to make the overall system even softer and more flexible than it was before. It was generous in what it shared, too, so a flourishing ecosystem grew around it, at least for the first few years. You could use pretty much any Twitter data to which you had access in any way you liked. It was a very simple, very powerful component, a tool rather than an environment or platform. In retrospect I wish we had used Twitter as a model when developing the Landing, rather than the kitchen sink approach that we settled for.

Twitter is widely viewed as a competitor to Facebook – increasingly even by the company itself – though it was (and still is, to an extent) a very different animal. Facebook has tried to emulate all of Twitter’s features as a subset of its own horrible evil mess, but completely misses the point. The strength of Twitter is that it (still) does one simple thing very well: it is primarily a hub that makes the rest of the Web more connected, rather than (like Facebook) sucking everything into it. However, that one simple thing is as soft and open to countless, unprestatable uses as an elastic band, a screwdriver, or good old fashioned email.  Jack Dorsey’s announcement of the new move itself is a classic example of this, creating a long-form announcement from short tweets. Beyond simply connecting stuff, people have used it to write novels, coordinate social protests, conduct personal conversations, influence elections, and thousands of other things. It is a very soft, very human-driven tool.

For a few years it was very open, and it seemed to be getting more so, but it lost its way after that and became much more the self-contained platform we see today, pulling a lot of features into its core, closing off many ways of connecting with and using it, and increasingly hardening things that should have stayed soft, notably in its algorithmic placing and sorting of tweets. Though its old character limit was frustrating at times, it was actually a very good idea to set such severe boundaries because it ensured that Twitter remained as a connecting hub, rather than a self-contained site. The new higher character limit is still somewhat constraining, but it makes longer-form conversations increasingly possible – especially when combined with the easy upload of video, files, images, etc – thus drawing people to stay more at the hub, rather than to visit the things that it connects. It has become more and more a social media platform, increasingly isolated, increasingly its own bubble, increasingly driven by the popularity contests and narcissism amplifiers that seldom end well. Twitter’s announcement, I hope, marks a reversal of this pattern. I hope (though don’t expect) that they get the Mastodon gang on board. I will watch with great interest, whatever happens.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/5140548/excellent-news-twitter-makes-a-bet-on-protocols-over-platforms

E-Learn 2019 presentation – X-literacies: beyond digital literacy

Here are  my slides from E-Learn 2019, in New Orleans. The presentation was about the nature of technologies and their roles in communities (groups, networks, sets, whatever), their highly situated nature, and their deep intertwingling with culture. In general it is an argument that literacies (as opposed to skills, knowledge, etc) might most productively and usefully be seen as the hard techniques needed to operate the technologies that are required for any given culture. As well as clarifying the term and using it in the same manner as the original term “literacy”, this implies there may be an indefinitely large range of literacies because we are all members of an indefinitely large number of overlapping cultures. All sorts of possibilities and issues emerge from this perspective.

Abstract: Dozens, if not hundreds, of literacies have been identified by academic researchers, from digital- to musical- to health- to network- literacy, as well as combinatorial terms like new-, multi-, 21st Century-, and media-literacy. Proponents seek ways to support the acquisition of such literacies but, if they are to be successful, we must first agree what we mean by ‘literacy’. Unfortunately, the term is used in many inconsistent and incompatible ways, from simple lists of skills to broad characteristics or tendencies that are either ubiquitous or meaninglessly vague. I argue that ‘literacy’ is most usefully thought of as the set of learned techniques needed to participate in the technologies of a given culture. Through use and application of a culture’s techniques, increasing literacy also leads to increasing knowledge of the associated facts and adoption of the values that come with that culture. Literacy is thus contextually situated, mutates over time as a culture and its technologies evolve, and participates in that co-evolution. As well as subsuming and eliminating much of the confusion caused by the proliferation of x-literacies, this opens the door to more accurately recognizing the literacies that we wish to use, promote and teach for any given individual or group.

 

Social Media Has Not Destroyed a Generation   – Scientific American

Well this is not a surprise. It turns out that social media and cellphone use have little to no effect on the mental well-being of teenagers. And, having just hung out with more than 10,000 young people in Vancouver, I’d say that they seem to be doing pretty well,

(if the video does not display, visit www.youtube.com/watch?v=ZdyBpfYvxs4).

Unfortunately, these wonderful young people are not to be confused with the very many utter creeps, idiots, paid lackeys of oil companies, bizarrely de-evolved evolution-deniers (not to mention climate-change deniers), and haters of all things decent who felt compelled to contribute to the live chat displayed alongside the YouTube video linked to above, as well as to far too many of the subsequent comments. This is what raw, unfiltered sets (the largely anonymous, non-networked social form that dominates on YouTube and many other social media) look like. The insane, the evil, and the stupid (often a mix of all three) have voices at least as loud as those who have something reasonable or human to say, and they have a platform where at least a few other people with ugly, broken souls will help them to feel validated, so they feel even more compelled to say the stupid, ugly, evil things they say. How dare they? Perhaps some of them are also children but, from many of the comments, I’d say that most have reached voting age. It’s not the kids that we need to worry about, apart from that they may be being brought up by such vile excuses for humanity, and that they have to learn to make sense of the stuff swamp of social media systems that enable such voices to be loudly heard.

When I hear Greta Thunberg talk it consistently brings tears to my eyes and sends shivers down my spine. She is astonishingly wonderful and deeply, deeply inspiring. She is brave, she is brilliant, she is right. She is not proposing anything apart from that politicians take action now on an unequivocal, plain to see, planet-wide threat, that is caused by problems that we know how to solve, and that demands political action. Yes, that will disrupt the lives of people that have profited from our collective madness – that is to say, most of us (but it is a hell of a lot less disruption than the alternative, at least for those not due to die any time soon). Yes, it is really difficult to make it happen. Yes, it means we will all have to change some of our ways, but that is no bad thing: our lives, and those of our children, and those of most of the living things on our planet, will be better as a result. And no, it is not her job to propose solutions, and she very deliberately does not try to do so, though she lives her life according to her convictions and does what she sees as necessary as an individual fighting the climate crisis. When she talks she simply states – with immense, infectious, intense passion – what is wrong, and demands that those who can fix it should do so. I am deeply humbled by this amazing teenager. We should all be.

Do buy the cheap, slim volume of her speeches, No One Is Too Small to Make a Difference. It is an inspiring book, and the proceeds will all go to charity.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/4949999/social-media-has-not-destroyed-a-generation-%C2%A0-scientific-american

Causal understanding is not necessary for the improvement of culturally evolving technology (paywalled)

https://www.nature.com/articles/s41562-019-0567-9

I’ve been struggling a bit with writing a chapter on how we should research technologies, especially soft technologies, in the light of their innate complexity, the difficulties of identifying relevant boundaries, their situated nature, the impossibility of identifying all possible uses for any soft technology (and the immense importance of the role of the user in their enactment), and the fact that they are far from fixed, amongst other things. This (regretably paywalled) paper helps support the theoretical model I am developing. Using an experimental method, it shows that technologies can be developed, and can gain in sophistication and complexity over multiple generations, without any of its designers having an accurate or complete understanding of how they work. It is particularly interesting when viewed through a lens of distributed/situated/extended cognition because of the role the technology itself plays in its evolution, and it accords very well with Kauffman’s notion of the adjacent possible and Arthur’s theory of technological evolution.

From the abstract…

“Here we show that a physical artefact becomes progressively optimized across generations of social learners in the absence of explicit causal understanding. Moreover, we find that the transmission of causal models across generations has no noticeable effect on the pace of cultural evolution. The reason is that participants do not spontaneously create multidimensional causal theories but, instead, mainly produce simplistic models related to a salient dimension. Finally, we show that the transmission of these inaccurate theories constrains learners’ exploration and has downstream effects on their understanding. These results indicate that complex technologies need not result from enhanced causal reasoning but, instead, can emerge from the accumulation of improvements made across generations.”

Originally posted at: https://landing.athabascau.ca/bookmarks/view/4846373/causal-understanding-is-not-necessary-for-the-improvement-of-culturally-evolving-technology-paywalled

My learning style

I am a visual, aural, read/write, kinaesthetic, introvert, extravert, sensing, intuitive, analytic, thinking, feeling, judging, perceiving, independent, dependent, collaborative, competitive, participant, avoidant, wholist, analytic, verbalizing, imaging, visualizing, deductive, synthetic, expansive, serialist, holist, field-dependent, field-independent, intrinsically motivated, extrinsically motivated, impulsive, reflexive, convergent, divergent, levelling, sharpening, concrete-sequential, concrete-random, abstract-sequential, abstract-random, assimilating, exploring, adaptive, innovative, reproductive, experiencing, thinking, doing, reflective, directed, self-directed, undirected, application-directed, meaning-directed, deep, surface, strategic, apathetic, elaborative, impulsive, concrete, independent, self-assertive, cerebral,  affective, type 1, type 2, type 3, global, scanning, focusing, physical, logical, social, solitary, musical-rhythmic, interpersonal, intrapersonal, spatial, body, active, common sense, dynamic, imaginative, quadrant 1, quadrant 2, quadrant 3, quadrant 4, theorizing, organizing, humanitarian, legislative, judicial, executive, tactile, pragmatic, versatile learner.

My birth sign is Aquarius, and I was born in the Year of the Rat.

Incidentally…

It appears that 97% of American teachers actually believe in learning styles, by which I mean the belief that there are persistent traits describing how people learn that can be used to determine the best way to teach them. This is despite at least most, if not all, of the many scores of such theories existing somewhere between astrology and fairies in terms of evidence for their relevance or applicability in real life learning. Though there may be ever-shifting conditions under which we may at times prefer one or other of whatever learning styles the theory we like offers – this may be a source of the persisting appeal of the idea – there is no reliable evidence that this is in any way relevant to whether or not we will learn better or worse (whatever we think that means) when offered a learning experience that is tailored to that preference. It’s not by any means for want of trying – countless studies exist, and that’s not counting probably many more that never saw the light of day because they had only null results to report and so were not deemed worthy of publication – so the obvious conclusion to be drawn is that these theories are most likely false.

It wouldn’t be so worrying were it not that there is evidence that such beliefs are harmful to learners and, even if there were not, then the time, effort, and money put into trying to use them would be far better spent on things that actually might work.

In the extremely unlikely event that it were one day proven that an individual has a persistent style of learning that, when we teach to that style, consistently leads to improved learning (however we measure that), then it would be my duty as a teacher to try to teach them to learn in other ways, because here’s the thing: the real world in which we are and must be lifelong learners doesn’t come neatly packaged in ways that fit your learning style. We can all learn to learn in all the ways that I list above, and then some, and we can all become better and smarter by applying the right strategy at the right time. We therefore need to cultivate as many diverse learning strategies as we can, and learn when to use them. That’s just common sense which, as it happens and surprisingly enough, is itself a learning style, according to the 4MAT model.

Signals, boundaries, and change: how to evolve an information system, and how not to evolve it

primitive cell development

For most organizations there tend to be three main reasons to implement an information system:

  1.     to do things the organization couldn’t do before
  2.     to improve things the organization already does (e.g. to make them more efficient/cheaper/better quality/faster/more reliable/etc)
  3.     to meet essential demands (e.g. legislation, keep existing apps working, etc)

There are other reasons (political, aesthetic, reputational, moral, corruption/bribery/kickbacks, familiarity, etc) but I reckon those are the main ones that matter. They are all very good reasons.

Costs and debts

With each IT solution there will always be costs, both initial and ongoing. Because we are talking about technology, and all technologies evolve to greater complexity over time, the ongoing costs will inevitably escalate. It’s not optional. This is what is commonly described as the ‘technological debt’ but that is a horrible misnomer. It is not a debt, but the price we pay for the solutions we need. If we don’t do it, our IT systems decay and die, starved of their connections with the evolving business and global systems around them. It’s no more of a debt than the need to eat or receive medical care is a debt for living.

Thinking locally, not globally

When money needs to be saved in an organization, senior executives tend to look at the inevitably burgeoning cost of IT and see it as ripe for pruning. IT managers thus tend to be placed under extreme pressure to ‘save’ costs. IT managers might often be relieved about that because they are almost certainly struggling to maintain the customized apps already, unless they have carefully planned for those increased costs over years (few do). Sensibly (from their own local perspective, given what they have been charged with doing), they therefore tend to strip out customizations, then shift to baseline applications, and/or cloud-based services that offer financial savings or, at least, predictable costs, giving the illusion of control. Often, they wind up firing, repurposing, or not renewing contracts for development staff, support staff, and others with deep knowledge of the old tools and systems. This keeps the budget in check so they achieve the goals set for them.

Unfortunately, assuming that the organization continues to need to do what it has been doing up to that point, the unavoidable consequence is that things that computers used to do are now done by people in the workforce instead. When made to perform hard mechanical tasks that computers can and should do, people are invariably far more fallible, slow, inconsistent, and inefficient. Far more. They tend to be reluctant, too. To make things worse, these mundane repetitive tasks take time, and crowd out other, more important things that people need to do, such as the things they were hired for. People tend to get tired, angry, and frustrated when made to do mechanical things over which they have little agency, which reduces productivity much further than simply the time lost in doing them. To make matters even worse, there is inevitably going to be a significant learning curve, during which staff try to figure out how to do the work of machines. This tends to lead to inflated training budgets (usually involving training sessions that, as decades of research show, are rarely very effective and that have to be repeated), time to read documentation, and more time taken out of the working day. Creativity, ingenuity, innovation, problem-solving, and interaction with others all suffer. The organization as a whole consequently winds up losing many times more (usually by orders of magnitude) than they saved on IT costs, though the IT budget now looks healthy again so it is often deemed to be a success. This is like taking the wheels off a car then proudly pointing to the savings in fuel that result. Unfortunately, such general malaises seldom appear in budget reports, and are rarely accounted for at all, because they get lost in the work that everyone is doing. Often, the only visible signs that it has happened are that the organization just gets slower, less efficient, less creative, more prone to mistakes, and less happy. Things start to break, people start to leave, sick days multiply. The reputation of the organization begins to suffer.
 
This is usually the point that more radical large scale changes to the organization are proposed, again usually driven by senior management who (unless they listen very carefully to what the workforce is telling them) may well attribute the problems they are seeing to the wrong causes, like external competition. A common approach to the problem is to impose more austerity, thus delivering the killing blow to an already demoralized workforce. That’s an almost guaranteed disaster. Another common way to tackle it is to take greater risks, made all the more risky thanks to having just converted creative, problem-solving, inquisitive workers into cogs in the machine, in the hope of opening up new sources of revenue or different goals. When done under pressure, that seldom ends well, though at least it has some chance of success, unlike austerity. This vicious cycle is hard to escape from. I don’t know of any really effective way to deal with it once it has happened.

Thinking in systems

The way to avoid it in the first place is not to kill off and directly replace custom IT solutions with baseline alternatives. There are very good reasons for almost all of those customizations that have almost certainly not gone away: all those I mentioned at the start of the post don’t suddenly cease to apply. It is therefore positively stupid to simply remove them without an extremely deep, multifaceted analysis of how they are used and who uses them, and even then with enormous conservatism and care. However, you probably still want to get rid of them eventually anyway, because, as well as being an ever-increasing cost,  they have probably become increasingly out of line with how the organization and the world around it is evolving. Unless there has been a steady increase in investment in new IT staff (too rare), so much time is probably now spent keeping old systems going that there is no time to work on improvements or new initiatives. Unless more money can be put into maintaining them (a hard sell, though important to try) the trick is not to slash and burn, and definitely not to replace old customized apps with something different and less well-tailored, but to gently evolve towards whatever long-term solution seems sensible using techniques such as those I describe below. This has a significant cost, too, but it’s not usually as high, and it can be spread over a much longer period.
 

For example…

If you wish to move away from reliance on a heavily customized learning management system to a more flexible and adaptive learning ecosystem made of more manageable pieces, the trick is to, first of all, build connectors into and out of your old system (if they do not already exist), to expose as many discrete services as possible, and then to make use of plugin hooks (or similar) to seamlessly replace existing functions with new ones. The same may well need to be done with the new system, if it does not already work that way. This is the most expensive part, because it normally demands development time, and what is developed will have to be maintained, but it’s worth it. What you are doing, at an abstract level, is creating boundaries around parts that can be treated as distinct (functions, components, objects, services, etc) and making sure that the signals that pass between them can be understood in the same way by subsystems on either side of the boundary.

Open industry standards (APIs, protocols, etc) are almost essential here, because apps at both sides of the boundary need to speak the same language. Proprietary APIs are risky: you do not want to start doing this then have a vendor decide to change its API or its terms and conditions. It’s particularly dangerous to do this with proprietary cloud-based services, where you don’t have any control whatsoever over APIs or backends,  and where sudden changes (sometimes without even a notification that they are happening) are commonplace. It’s fine to use containers or virtual machines in the cloud – they can be replaced with alternatives if things go wrong, and can be treated much like applications hosted locally – and it’s fine to use services with very well defined boundaries, with standards-based APIs to channel the signals. It is also fine to build your own, as long as you control both sides of the boundary, though maintenance costs will tend to be higher.  It is not fine to use whole proprietary applications or services in the cloud because you cannot simply replace them with alternatives, and changes are not under your control. Ideally, both old and new systems should be open source so that you are not bound to one provider, you can make any changes you need (if necessary), and you can rely on having ongoing access to older versions if things change too fast.
 
Having done this, you have two main ways to evolve, that you can choose according to needs:

  1.  to gradually phase in the new tools you want and phase out the old ones you don’t want in the old system until, like the ship of Theseus, you have replaced the entire thing. This lets you retain your customizations and existing investments (especially in knowledge of those systems) for the longest time, because you can replace the parts that do not rely on them before tackling those that do. Meanwhile, those same fresh tools can start to make their appearance in whatever other new systems you are trying to build, and you can make a graceful, planned transition as and when you are ready. This is particularly useful if there is a great deal of content and learning already embedded in the system, which is invariably the case with LMSs. It means people can mostly continue to work the way they’ve always worked, while slowly learning about and transitioning to a new way of working.
  2.  to make use of some services provided by the old system to power the new one. For instance, if you have a well-established means of generating class lists or collecting assessment data that involves a lot of custom code, you can offer that as a service from the old tool to your new tool, rather than reimplementing it afresh straight away or requiring users to manually replace the custom functions with fallible human work. Eventually, once the time is right to move and you can afford it, you can then simply replace it with a different service, with virtually no disruption to anyone. This is better when you want a clean break, especially useful when the new system does things that the original could not do, though it still normally allows simultaneous operation for a while if needed, as well as the option to fall back to the old system in the event of a disaster.

There are other hybrid alternatives, such as setting up other systems to link both, so that the systems do not interact directly but via a common intermediary. In the case of an LMS migration, this might be a learning record store (LRS) or student record system, for instance. The general principle, though, is to keep part or all of the old system running simultaneously for however long it is needed, parcellating its tools and services, while slowly transitioning to the new. Of course, this does imply extra cost in the short term, because you now have to manage at least two systems instead of one. However, by phasing it this way you greatly reduce risk, spread costs over a timeframe that you control, and allow for changes in direction (including reversal) along the way, which is always useful. The huge costs you save are those that are hidden from conventional accounting – the time, motivation, and morale of the workforce that uses the system. As a useful bonus, this service-oriented approach to building your systems also allows you to insert other new tools and implement other new ideas with a greatly diminished level of risk, with fewer recurring costs, and without the one-time investment of having to deal with your whole monolithic codebase and data. This is great if you want to experiment with innovations at scale. Once you have properly modularized your system, you can grow it and change it by a process of assembly. It often allows you to offer more control to end users, too: for instance, in our LMS example you might allow individuals to choose between different approaches to a discussion forum, or content presentation, or to insert a research-based component without so many of the risks (security, performance, reliability, etc) normally associated with implementing less well-managed code.

Signals and boundaries

In essence, this is all about signals and boundaries. The idea is to identify and, if they don’t exist, create boundaries between distinct parts of systems, then to focus all your management efforts on the signals that pass across them. As long as the signals remain the same from both sides, what lies on either side of the boundaries can be isolated and replaced when needed. This happens to be the way that natural systems mainly evolve too, from organisms to ecosystems. It has done pretty good service for a good billion years or so.

 
 

Education for life or Education for work? Reflections on the RBC Future Skills Report

Tony Bates extensively referenced this report from the Royal Bank of Canada on Canadian employer demands for skills over the next few years, in his characteristically perceptive keynote at CNIE 2019 last week (it’s also referred to in his most recent blog post). It’s an interesting read. Central to its many findings and recommendations are that the Canadian education system is inadequately designed to cope with these demands and that it needs to change. The report played a big role in Tony’s talk, though his thoughts on appropriate responses to that problem were independently valid in and of themselves, and not all were in perfect alignment with the report. Tony Bates at CNIE 2019

The 43-page manifesto (including several pages of not very informative graphics) combines some research findings, with copious examples to illustrate its discoveries, and with various calls to action based on them. I guess not surprisingly for a document intended to ignite, it is often rather hard to tell in any detail how the research itself was conducted. The methodology section is mainly on page 33 but it doesn’t give much more than a broad outline of how the main clustering was performed, and the general approach to discovering information. It seems that a lot of work went into it, but it is hard to tell how that work was conducted.

A novel (-ish) finding: skillset clusters

Perhaps the most distinctive and interesting research discovery in the report is a predictive/descriptive model of skillsets needed in the workplace. By correlating occupations from the federal NOC (National Occupational Classification) with a US Labor Department dataset (O*NET) the researchers abstracted and identified six distinct clusters of skillsets, the possessors of which they characterize as:

  • solvers (engineers, architects, big data analysts, etc)
  • providers (vets, musicians, bloggers, etc)
  • facilitators (graphic designers, admin assistants, Uber drivers, etc)
  • technicians (electricians, carpenters, drone assemblers, etc)
  • crafters (fishermen, bakers, couriers, etc)
  • doers (greenhouse workers, cleaners, machine-learning trainers, etc)

From this, they make the interesting, if mainly anecdotally supported, assertion that there are clusters of occupations across which these skills can be more easily transferred. For instance, they reckon, a dental assistant is not too far removed from a graphic designer because both are high on the facilitator spectrum (emotional intelligence needed). They do make the disclaimer that, of course, other skills are needed and someone with little visual appreciation might not be a great graphic designer despite being a skilled facilitator. They also note that, with training, education, apprenticeship models, etc, it is perfectly possible to move from one cluster to another, and that many jobs require two or more anyway (mine certainly needs high levels of all six). They also note that social skills are critical, and are equally important in all occupations. So, even if their central supposition is true, it might not be very significant.

There is a somewhat intuitive appeal to this, though I see enormous overlap between all of the clusters and find some of the exemplars and descriptions of the clusters weirdly misplaced: in what sense is a carpenter not a crafter, or a graphic designer not a provider, or an electrician not a solver, for instance? It treads perilously close to the borders of x-literacies – some variants of which come up with quite similar categories – or learning style theories, in its desperate efforts to slot the world into manageable niches regardless of whether there is any point to doing so. The worst of these is the ‘doers’ category, which seems to be a lightly veiled euphemism for ‘unskilled’ (which, as they rightly point out, relates to jobs that are mostly under a great deal of threat). ‘Doing’ is definitely ripe for transfer between jobs because mindless work in any occupation needs pretty much the same lack of skill. My sense is that, though it might be possible to see rough patterns in the data, the categories are mostly very fuzzy and blurred, and could easily be used to label people in very unhelpful ways. It’s interesting from a big picture perspective, but, when you’re applying it to individual human beings, this kind of labelling can be positively dangerous. It could easily lead to a species of the same general-to-specific thinking that caused the death of many airplane pilots prior to the 1950s, until the (obvious but far-reaching) discovery that there is no such thing as an average-sized pilot. You can classify people into all sorts of types, but it is wrong to make any further assumptions about them because you have done so. This is the fundamental mistake made by learning style theorists: you can certainly identify distinct learner types or preferences but that makes no difference whatsoever to how you should actually teach people.

Education as a feeder for the job market

Perhaps the most significant and maybe controversial findings, though, are those leading more directly to recommendations to the educational and training sector, with a very strong emphasis on preparedness for careers ahead. One big thing bothers me in all of this. I am 100% in favour of shifting the emphasis of educational institutions from knowledge acquisition to more fundamental and transferable capabilities: on that the researchers of this report hit the nail on the head. However, I don’t think that the education system should be thought of, primarily, as a feeder for industry or preparation for the workplace. Sure, it’s definitely one important role for education, but I don’t think it’s the dominant one, and it’s very dangerous indeed to make that its main focus to the exclusion of the rest. Education is about learning to be a human in the context of a society; it’s about learning to be part of that culture and at least some of its subcultures (and, ideally, about understanding different cultures). It’s a huge binding force, it’s what makes us smart, individually and collectively, and it is by no means limited to things we learn in institutions or organizations. Given their huge role in shaping how we understand the world,  at the very least media (including social media) should, I think, be included whenever we talk of education. In fact, as Tony noted, the shift away from institutional education is rapid and on a vast scale, bringing many huge benefits, as well as great risks. Outside the institutions designed for the purpose, education is often haphazard, highly prone to abuse, susceptible to mob behaviours, and often deeply harmful (Trump, Brexit, etc being only the most visible tips of a deep malaise). We need better ways of dealing with that, which is an issue that has informed much of my research. But education (whether institutional or otherwise) is for life, not for work.

I believe that education is (and should be) at least partly concerned with passing on what we know, who we have been, who we are, how we behave, what we value, what we share, how we differ, what drives us, how we matter to one another. That is how it becomes a force for societal continuity and cohesion, which is perhaps its most important role (though formal education’s incidental value to the economy, especially through schools, as a means to enable parents to work cannot be overlooked). This doesn’t have to exclude preparation for work: in fact, it cannot.  It is also about preparing people to live in a culture (or cultures), and to continue to learn and develop productively throughout their lives, evolving and enhancing that culture, which cannot be divorced from the tools and technologies (including rituals, norms, rules, methods, artefacts, roles, behaviours, etc) of which the cultures largely consist, including work. Of course we need to be aware of, and incorporate into our teaching, some of the skills and knowledge needed to perform jobs, because that’s part of what makes us who we are. Equally, we need to be pushing the boundaries of knowledge ever outwards to create new tools and technologies (including those of the arts, the humanities, the crafts, literature, and so on, as well as of sciences and devices) because that’s how we evolve. Some – only some – of that will have value to the economy. And we want to nurture creativity, empathy, social skills, communication skills, problem-solving skills, self-management skills, and all those many other things that make our culture what it is and that allow us to operate productively within it, that also happen to be useful workplace skills. But human beings are also much more than their jobs. We need to know how we are governed, the tools needed to manage our lives, the structures of society. We need to understand the complexities of ethical decisions. We need to understand systems, in all their richness. We need to nurture our love of arts, sports, entertainment, family life, the outdoors, the natural and built environment, fine (and not fine) dining, being with friends, talking, thinking, creating stuff, appreciating stuff, and so on. We need to develop taste (of which Hume eloquently wrote hundreds of years ago).  We need to learn to live together. We need to learn to be better people. Such things are (I think) more who we are, and more what our educational systems should focus on, than our productive roles in an economy. The things we value most are, for the most part, seldom our economic contributions to the wealth of our nation, and the wealth of a nation should never be measured in economic terms.  Even those few that love money the most usually love the power it brings even more, and that’s not the same thing as economic prosperity for society. In fact, it is often the very opposite.

I’m not saying economic prosperity is unimportant, by any means: it’s often a prerequisite for much of the rest, and sometimes (though far from consistently) a proxy marker for them. And I’m not saying that there is no innate value in the process of achieving economic prosperity: many jobs are critical to sustaining that quality of life that I reckon matters most, and many jobs actually involve doing the very things we love most. All of this is really important, and educational systems should cater for it. It’s just that future employment should not be thought of as the main purpose driving education systems.

Unfortunately, much of our teaching actually is heavily influenced by the demands of students to be employable, heavily reinforced on all sides by employers, families, and governments, and that tends to lead to a focus on topics, technical skillsets, and subject knowledge, not so much to the exclusion of all the rest, but as the primary framing for it. For instance, HT to Stu Berry and Terry Anderson for drawing my attention to the mandates set by the BC government for its post secondary institutions, that are a litany of shame, horribly focused on driving economic prosperity and feeding industry, to the exclusion of almost anything else (including learning and teaching, or research for the sake of it, or things that enrich us as human beings rather than cogs in an economic machine). This report seems to take the primary role of education as a driver of economic prosperity as just such a given. I guess, being produced by a bank, that’s not too surprising, but it’s worth viewing it with that bias in mind.

And now the good news

What is heartwarming about this report, though, is that employers seem to want (or think they will want) more or less exactly those things that also enrich our society and our personal lives. Look at this fascinating breakdown of the skills employers think they will need in the future (Tony used this in his slides):

Projected skills demands, from the RBC future skills report

 

There’s a potential bias due to the research methodology, that I suspect encouraged participants to focus on more general skills, but it’s really interesting to see what comes in the first half and what dwindles into unimportance at the end.

Topping the list are active listening, speaking, critical thinking, comprehension, monitoring, social perceptiveness, coordination, time management, judgement and decision-making, active learning, service orientation, complex problem solving, writing, instructing, persuasion, learning strategies, and so on. These mostly quite abstract skills (in some cases propensities, albeit propensities that can be cultivated) can only emerge within a context, and it is not only possible but necessary to cultivate them in almost any educational intervention in any subject area, so it is not as though they are being ignored in our educational systems. More on that soon. What’s interesting to me is that they are the human things, the things that give us value regardless of economic value. I find it slightly disconcerting that ethical or aesthetic sensibilities didn’t make the list and there’s a surprising lack of mention of physical and mental health but, on the whole, these are life skills more than just work skills.

Conventional education can and often does cultivate these skills. I am pleased to brag that, as a largely unintentional side-effect of what I think teaching in my fields should be about, these are all things I aim to cultivate in my own teaching, often to the virtual exclusion of almost everything else. Sometimes I have worried (a little) that I don’t have very high technical expectations of my students. For instance, my advanced graduate level course in information management provides technical skills in database design and analysis that are, for the most part, not far above high-school level (albeit that many students go far beyond that); my graduate level social computing course demands no programming skills at all (technically, they are optional); my undergraduate introduction to web programming course sometimes leads to limited programming skills that would fail to get them a passing grade in a basic computer science course (though they typically pass mine). However (and it’s a huge HOWEVER) they have a far greater chance to acquire far more of those skills that I believe matter, and (gratifyingly) employers seem to want, than those who focus only on mastery of the tools and techniques. My web programming students produce sites that people might actually want to visit, and they develop a vast range of reflective, critical thinking, complex problem-solving, active learning, judgment, persuasion, social perceptiveness and other skills that are at the top of the list. My information management students get all that, and a deep understanding of the complex, social, situated nature of the information management role, with some notable systems analysis skills (not so much the formal tools, but the ways of understanding and thinking in systems). My social computing students get all that, and come away with deep insights into how the systems and environments we build affect our interactions with one another, and they can be fluent, effective users and managers of such things. All of the successful ones develop social and communication skills, appropriate to the field. Above all, my target is to help students to love learning about the subjects of my courses enough to continue to learn more. For me, a mark of successful teaching is not so much that students have acquired a set of skills and knowledge in a domain but that they can, and actually want to, continue to do so, and that they have learned to think in the right ways to successfully accomplish that. If they have those skills, then it is not that difficult to figure out specific technical skillsets as and when needed. Conveniently, and not because I planned it that way, that happens to be what employers want too.

Employers don’t (much) want science or programming skills: so what?

Even more interesting, perhaps, than the skills employers do want are the skills they do not want, from Operation Monitoring onwards in the list, that are often the primary focus of many of our courses. Ignoring the real nuts and bolts stuff at the very bottom like installation, repairing, maintenance, selection (more on that in a minute), it is fascinating that skills in science, programming, and technology design are hardly wanted at all by most companies, but are massively over-represented in our teaching. The writers of the report do offer the proviso that it is not impossible that new domains will emerge that demand exactly these skills but, right now and for the foreseeable future, that’s not what matters much to most organizations. This doesn’t surprise me at all. It has long been clear that the demand for people that create the foundations is, of course, going to be vastly much smaller than the demand for people that build upon them, let alone the vastly greater numbers that make use of what has been built upon them. It’s not that those skills are useless – that’s a million miles from the truth – but that there is a very limited job market for them. Again, I need to emphasize that educators should not be driven by job markets: there is great value in knowing this kind of thing regardless of our ability to apply it directly in our jobs. On the other hand, nor should we be driven by a determination to teach all there is to know about foundations, when what interests people (and employers, as it happens) is what can be done with them. And, in fact, even those building such foundations desperately need to know that too, or the foundations will be elegant but useless. Importantly, those ‘foundational’ skills are actually often anything but, because the emergent structures that arise from them obey utterly different rules to the pieces of which they are made. Knowing how a cell works tells you nothing whatsoever about function of a heart, let alone how you should behave towards others, because different laws and principles apply at different levels of organization. A sociologist, say, really doesn’t need to know much about brain science, even though our brains probably contribute a lot to our social systems, because it’s the wrong foundation, at the wrong level of detail. Similarly, there is not a lot of value in knowing how CPUs work if your job is to build a website, or a database system supporting organizational processes (it’s not useless, but it’s not very useful so, given limited resources, it makes little sense to focus on it). For almost all occupations (paid or otherwise) that make use of science and technology, it matters vastly much more to understand the context of use, at the level of detail that matters, than it does to understand the underlying substructures. This is even true of scientists and technologists themselves: for most scientists, social and business skills will have a far greater effect on their success than fundamental scientific knowledge. But, if students are interested in the underlying principles and technologies on which their systems are based, then of course they should have freedom and support to learn more about them. It’s really interesting stuff, irrespective of market demand. It enriches us. Equally, they should be supported in discovering gothic literature, social psychology, the philosophy of art, the principles of graphic design, wine making, and anything else that matters to them. Education is about learning to be, not just learning to do. Nothing of what we learn is wasted or irrelevant. It all contributes to making us creative, engaged, mutually supportive human beings.

With that in mind, I do wonder a bit about some of the skills at the bottom of the list. It seems to me that all of the bottom four demand – and presuppose – just about all of those in the top 12. At least, they do if they are done well. Similarly for a few others trailing the pack. It is odd that operation monitoring is not much desired, though monitoring is. It is strange that troubleshooting is low in the ranks, but problem-solving is high. You cannot troubleshoot without solving problems. It’s fundamental. I guess it speaks to the idea of transferability and the loss of specificity in roles. My guess is that, in answering the questions of the researchers, employers were hedging their bets a bit and not assuming that specific existing job roles will be needed. But conventional teachers could, with some justification, observe that their students are already acquiring the higher-level, more important skills, through doing the low-level stuff that employers don’t want as much. Though I have no sympathy at all with our collective desire to impose this on our students, I would certainly defend our teaching of things that employers don’t want, at least partly because (in the process) we are actually teaching far more. I would equally defend even the teaching of Latin or ancient Greek (as long as these are chosen by students, never when they are mandated) because the bulk of what students learn is never the skill we claim to be teaching. It’s much like what the late, wonderful, and much lamented Randy Pausch called a head fake – to be teaching one thing of secondary importance while primarily teaching another deeper lesson – except that rather too many teachers tend to be as deceived as their students as to the real purpose and outcomes of their teaching.

Automation and outsourcing

As the report also suggests, it may also be that those skills lower in the ranking tend to be things that can often be outsourced, including (sooner or later) to machines. It’s not so much that the jobs will not be needed, but that they can be either automated or concentrated in an external service provider, reducing the overall job market for them. Yes, this is true. However, again, the methodology may have played a large role in coming to this conclusion. There is a tendency of which we are all somewhat guilty to look at current patterns of change (in this case the trend towards automation and outsourcing) and to assume that they will persist into the future. I’m not so sure.

Outsourcing

Take the stampede to move to the cloud, for instance, which is a clear underlying assumption in at least the undervaluing of programming. We’ve had phases of outsourcing several times before over the past 50 or 60 years of computing history. Cloud outsourcing is only new to the extent that the infrastructure to support it is much cheaper and more well-established than it was in earlier cycles, and there are smarter technologies available, including many that benefit from scale (e.g. AI, big data). We are currently probably at or near peak Cloud, but it is just a trend even if it has yet to peak. It might last a little longer than the previous generations (which, of course, never actually went away – it’s just an issue of relative dominance) but it suffers from most of the problems that brought previous outsourcing hype cycles to an end. The loss of in-house knowledge, the dangers of proprietary lock-in, the surrender of control to another entity that has a different (and, inevitably, at some point conflicting) agenda, and so on, are all counter forces to hold outsourcing in check. History and common sense suggests that there will eventually be a reversal of the trend and, indeed, we are seeing it here and there already, with the emergence of private clouds, regional/vertical cloud layers, hybrid clouds, and so on. Big issues of privacy and security are already high on the agendas of many organizations, with an increasing number of governments starting to catch up with legislation that heavily restricts unfettered growth of (especially) US-based hosting, with all the very many very bad implications for privacy that entails. Increasingly, businesses are realizing that they have lost the organizational knowledge and intelligence to effectively control their own systems: decisions that used to be informed by experts are now made by middle-managers with insufficient detailed understanding of the complexities, who are easy prey for cloud companies willing to exploit their ignorance. Equally, they are liable to be flanked by those who can adapt faster and less uniformly, inasmuch as everyone gets the same tools in the Cloud so there is less to differentiate one user of it from the next. OK, I know that is a sweeping generalization – there are many ways to use cloud resources that do not rely on standard tools and services. We don’t have to buy in to the proprietary SaaS rubbish, and can simply move servers to containers and VMs while retaining control, but the cloud companies are persuasive and keen to lure us in, with offers of reduced costs, higher reliability, and increased, scalable performance that are very enticing to stressed, underfunded CIOs with immediate targets to meet. Right now, cloud providers are riding high and making ridiculously large profits on it, but the same was true of IBM (and its lesser competitors) in the 60s and 70s. They were brought down (though never fully replaced) by a paradigm change that was, for the most part, a direct reaction to the aforementioned problems, plus a few that are less troublesome nowadays, like performance and cost of leased lines. I strongly suspect something similar will happen again in a few years.

Automation and the end of all things we value

Automation – especially through the increased adoption of AI techniques – may be a different matter. It is hard to see that becoming less disruptive, albeit that the reality is and will be much more mundane than the hype, and there will be backlashes. However, I greatly fear that we have a lot of real stupidity yet to come in this. Take education, for instance. Many people whose opinions I otherwise respect are guilty of thinking that teachers can be, to a meaningful extent, replaced by chatbots. They are horribly misguided but, unfortunately, people are already doing it, and claiming success, not just in teaching but in fooling students that they are being taught by a real teacher.  You can indeed help people to pass tests through the use of such tools. However, the only things that tests prove about learning is that you have learned to pass them. That’s not what education is for. As I’ve already suggested, education is really not much to do with the stuff we think we teach. It is about being and becoming human. If we learn to be human from what are, in fact, really very dumb machines with no understanding whatsoever of the words they speak, no caring for us, no awareness of the broader context of what they teach, no values to speak of at all, we will lower the bar for artificial intelligence because we will become so much dumber ourselves. It will be like being taught by an unusually tireless and creepily supportive (because why would you train a system to be otherwise?) person. We should not care for them, and that matters, because caring (both ways) is critical to the relationship that makes learning with others meaningful. But it will be even worse if and when we do start caring for them (remember the Tamagotchi?).  When we start caring for soulless machines (I don’t mean ‘soul’ in a religious or transcendent sense), when it starts to matter to us that we are pleasing them, we will learn to look at one another in the same way and, in the process, lose our own souls.  A machine, even one that fools us it is human, makes a very poor role model. Sure, let them handle helpdesk enquiries (and pass them on if they cannot help), let them supplement our real human interactions with useful hints and suggestions, let them support us in the tasks we have to perform, let them mark our tests to double-check we are being consistent: they are good at that kind of thing, and will get better. But please, please, please don’t let them replace teachers.

I am afraid of AI, not because I am bothered by the likelihood of an AGI (artificial general intelligence) superseding our dominant role on the planet: we have at least decades to think about that, and we can and will augment ourselves with dumb-but-sufficient AI to counteract any potential ill effects. The worst outcome of AI in the foreseeable future is that we devalue ourselves, that we mistake the semblance of humanity for humanity itself, that machines will become our role models. We may even think they are better than us, because they will have fewer human foibles and a tireless, on-demand, semblance of caring that we will mistake for being human (a bit like obsequious serving staff seeking tips in a restaurant, but creepier, less transparent, and infinitely patient). Real humans will disappoint us. Bots will be trained to be what their programmers perceive as the best of us, even though we don’t have more than the glimmerings of an idea of what ‘best’ actually means (philosophers continue to struggle with this after thousands of years, and few programmers have even studied philosophy at a basic level). That way the end of humanity lies: slowly, insidiously, barely noticeably at first. Not with a bang but with an Alicebot. Arthur C. Clark delightfully claimed that any teacher who could be replaced by a machine should be. I fear that we are not smart enough to realize that it is, in fact, very easy to successfully replace a teacher with a machine if you don’t understand the teacher’s true role in the educational machine, and you don’t make massive changes to it. As long as we think of education as the achievement of pre-specified outcomes that we measure using primitive tools like standardized tests, exams, and other inauthentic metrics, chatbots will quite easily supersede us, despite their inadequacies. It is way too easy to mistake the weirdly evolved educational system that we are part of for education itself: we already do so in countless ways. Learning management systems, for instance, are not designed for learning: they are designed to replicate mediaeval classrooms, with all the trimmings, yet they have been embraced by nearly all institutions because they fit the system. AI bots will fit even better. If we do intend to go down this path (and many are doing so already) then please let’s think of these bots as supplemental, first line support, and please let’s make it abundantly clear that they are limited, fixed-purpose mechanisms, not substitutes but supplements that can free us from trivial tasks to let us concentrate on being more human.

Co-ops and placements

The report makes a lot of recommendations, most of which make sense – e.g. lifelong support for learning from governments, focus on softer more flexible skills, focus on adaptability, etc. Notable among these is the suggestion, as one of its calls to action, that all PSE students should engage in some form of  meaningful work-integrated learning placements during their studies. This is something that we have been talking about offering to our program students in computing for some time at Athabasca University, though the demand is low because a large majority of our students are already working while studying, and it is a logistical nightmare to do this across the whole of Canada and much of the rest of the globe. Though some AU programs embed it (nursing, for instance) I’m not sure we will ever get round to it in computing. I do very much agree that co-ops and placements are typically a good idea for (at least) vocationally-oriented students in conventional in-person institutions. I supervised a great many of these (for computing students) at my former university and observed the extremely positive effects it usually had, especially on those taking the more humanistic computing programs like information systems, applied computing, computer studies, and so on. When they came back from their sandwich year (UK terminology), students were nearly always far wiser, far more motivated, and far more capable of studying than the relatively few that skipped the opportunity. Sometimes they were radically transformed – I saw borderline-fail students turn into top performers more than once – but, apart from when things fell apart (not common, but not unheard of), it was nearly always worth far more than at least the previous couple of years of traditional teaching. It was expensive and disruptive to run, demanding a lot from all academic staff and especially from those who had to organize it all, but it was worth it.

But, just because it works in conventional institutions doesn’t mean that it’s a good idea. It’s a technological solution that works because conventional institutions don’t. Let’s step back a bit from this for a moment. Learning in an authentic context, when it is meaningful and relevant to clear and pressing needs, surrounded by all the complexities of real life (notwithstanding that education should buffer some of that, and make the steps less risky or painful), in a community or practice, is a really good idea. Apprenticeship models have thousands of years of successful implementation to prove their worth, and that’s essentially what co-ops or placements achieve, albeit only in a limited (typically 3-month to 1 year) timeframe. It’s even a good idea when the study area and working practices do not coincide, because it allows many more connections to be made in both aspects of life. But why not extend that to all (or almost all) of the process? To an extent, this is what we at Athabasca already do, although it tends to be more the default context than something we take intentional advantage of. Again, my courses are an exception – most of mine (and all to some extent) rely on students having a meaningful context of their own, and give opportunities to integrate work or other interests and study by default. In fact, one of the biggest problems I face in my teaching arises on those rare occasions when students don’t have sufficient aspects of work or leisure that engage them (e.g. prisoners or visiting students from other universities), or work in contexts that cannot be used (e.g. defence workers). I have seen it work for in-person contexts, too: the Teaching Company Scheme in the UK, that later became Knowledge Transfer Partnerships, has been hugely successful over several decades, marrying workplace learning with academic input, usually leading to a highly personalized MSc or MA while offering great benefits to lecturers, employers and students alike. They are fun, but resource-intensive, to supervise. Largely for this reason, in the past it might have been hard to make this scalable to lower than graduate levels of learning, but modern technologies – shared workspaces, blogs, portfolio management tools, rich realtime meeting tools, etc, and a more advanced understanding of ways to identify and record competencies – make it far more possible. It seems to me that what we want is not co-ops or placements, but a robust (and, ideally, publicly funded) approach to integrating academic and in-context learning. Already, a lot of my graduate students and a few undergraduates are funded by their employers, working on our courses at the same time as doing their existing jobs, which seems to benefit all concerned, so there’s clearly a demand. And it’s not just an option for vocational learning. Though (working in computing) much of my teaching does have a vocational grounding, if not a vocational focus, I have come across students elsewhere across the university who are doing far less obviously job-related studies with the support of their employers. In fact, it is often a much better idea for students to learn stuff that is not directly applicable to their workplace, because the boundary-crossing it entails better improves a vast range of the most important skills identified in the RBC report – creativity, communication, critical thinking, problem solving, judgement, listening, reading, and so on. Good employers see the value in that.

Conclusions

Though this is a long post, I have only cherry-picked a few of the many interesting issues that emerge from the report, but I think there are some general themes in my reactions to it that are consistent:

1: it’s not about money

Firstly, the notion that educational systems should be primarily thought of as feeders for industry is dangerous nonsense. Our educational systems are preparation for life (in society and its cultures), and work is only a part of that. Preparedness for work is better seen as a side-effect of education, not its purpose. And education is definitely not the best vehicle for driving economic prosperity. The teaching profession is almost entirely populated by extremely smart, capable, people who (especially in relation to their qualifications) are earning relatively little money. To cap it all, we often work longer hours, in poorer conditions than many of our similarly capable industry colleagues. Though a fair living wage is, of course, very important to us, and we get justly upset when offered unfair wages or worsening conditions, we don’t work for pay: we are paid for our work. Notwithstanding that a lack of money is a very bad thing indeed and should be avoided like the plague, we do so precisely because we think there are some things – common things –  that are much more important than money (this may also partly account for a liberal bias in the profession, though it also helps that the average IQ of teachers is a bit above the norm). And, whether explicitly or otherwise, this is inevitably part of what we teach. Education is not primarily about learning a set of skills and facts: it’s about learning to be, and the examples that teachers set, the way they model roles, cannot help but come laden with their own values. Even if we scrupulously tried to avoid it, the fact of our existence serves as a prime example of people who put money relatively low on their list of priorities. If we have an influence (and I hope we do) we therefore encourage people to value things other than a large wage packet. So, if you are going to college or school in the hope of learning to make loads of money, you’re probably making the wrong choice. Find a rich person instead and learn from them.

2: it is about integrating education and the rest of our lives

Despite its relentless focus on improving the economy, I think this report is fundamentally right in most of the suggestions it makes about education, though it doesn’t go far enough. It is not so much that we should focus on job-related skills (whatever they might be) but that we should integrate education with and throughout our lives. The notion of taking someone out of their life context and inflicting a bunch of knowledge-acquisition tasks with inauthentic, teacher-led criteria for success, not to mention to subjugate them to teacher control over all that they do, is plain dumb. There may be odd occasions where retreating from and separating education from the world is worthwhile, but they are few and far between, and can be catered for on an individual needs basis.

Our educational processes evolved in a very different context, where the primary intent was to teach dogma to the many by the few, and where physical constraints (rarity of books/reading skills, limited availability of scholars, limits of physical spaces) made lecture forms in dedicated spaces appropriate solutions to those particular technical problems. Later, education evolved to focus more on creating a pliant and capable workforce to meet the needs of employers and the military, which happened to fit fairly well with the one-to-many top-down-control models devised to teach divinity etc. Though those days are mostly ended, we still retain strong echoes of these roles in much of our structure and processes – our pedagogies are still deeply rooted in the need to learn specific stuff, dictated and directed by others, in this weird, artificial context. Somehow along the way (in part due to higher education, at least, formerly being a scarce commodity) we turned into filters and gatekeepers for employment purposes.  But, today, we are trying to solve different problems. Modern education has tended to tread a shifting path between supporting individual development and improving our societies: these should be mutually supportive roles though different educational systems tend to put more emphasis on one than the other. With that in mind, it no longer makes sense to routinely (in fact almost universally) take people out of their physical, social, or work context to learn stuff. There are times that it helps or may even be necessary: when we need access to expensive shared resources (that mediaeval problem again), for instance, or when we need to work with in-person communities (hard to teach acting unless you have an opportunity to act with other actors, for example), or when it might be notably dangerous to practice in the real world (though virtual simulations can help). But, on the whole, we can learn far better when we learn in a real world context, where we can put our learning directly into useful practice, where it has value to us and those around us. Community matters immensely – for learning, for motivation, for diversity of ideas, for belonging, for connection, etc – and one of the greatest values in traditional education is that it provides a ready-made social context. We should not throw the baby out with the bathwater and it is important to sustain such communities, online or in-person. But it does not have to be, and should not ever be, the only social context, and it does not need to be the main social context for learning. Pleasingly, in his own excellent keynote at CNIE, our president Neil Fassina made some very similar points. I think that Athabasca is well on course towards a much brighter future.

3: what we teach is not what you learn

Finally, the whole education system (especially in higher education) is one gigantic head fake. By and large, the subjects we teach are of relatively minor significance. We teach ways of thinking, we teach values, we teach a few facts and skills, but mainly we teach a way of being. For all that, what you actually learn is something else entirely, and it is different from what every one of your co-learners learns, because 1) you are your main and most important teacher and 2) you are surrounded by others (in person, in artefacts they create, online) who also teach you. We need to embrace that far more than we typically do. We need to acknowledge and celebrate the differences in every single learner, not teach stuff at them in the vain belief that what we have to tell you matters more than what you want to learn, or that somehow (contrary to all evidence) everyone comes in and leaves knowing the same stuff. We’ve got to stop rewarding and punishing compliance and non-compliance.

What you learn changes you. It makes you able to see things differently, do things differently, make new connections. Anything you learn. There is no such thing as useless learning. It is, though, certainly possible to learn harmful things – misconceptions, falsehoods, blind beliefs, and so on – so the most important skill is to distinguish those from the things that are helpful (not necessarily true – helpful). On the whole, I don’t like approaches to teaching that make you learn stuff faster (though they can be very useful when solving some kinds of problem) because it devalues the journey. I like approaches that help you learn better: deeper, more connected, more transformative. This doesn’t mean that the RBC report is wrong in criticizing our current educational systems, but it is wrong to believe that the answer is to stop (or reduce) teaching the stuff that employers don’t think they need. Learners should learn whatever they want or need to learn, whenever they need to do so, and educational institutions (collectively) should support that. But that also doesn’t mean teachers should teach what learners (or employers, or governments) think they should teach, because 1) we always teach more than that, whether we want to or not, and it all has value and 2) none of these entities are our customers. The heartbreaking thing is that some of the lessons most of us unintentionally teach – from mindless capitulation to authority, to the terrible approaches to learning nurtured by exams, to the truly awful beliefs that people do not like/are not able to learn certain subjects or skills – are firmly in the harmful category.  It does mean that we need to be more aware of the hidden lessons, and of what our students are actually learning from them. We need to design our teaching in ways that allow them to make it relevant and meaningful in their lives. We need to design it so that every student can apply their learning to things that matter to them, we need to help them to reflect and connect, to adopt approaches, attitudes, and values that they can constantly use throughout their lives, in the workplace or not. We need to help them to see what they have learned in a broader social context, to pay it forward and spread their learning contagiously, both in and out of the classroom (or wherever they are doing their learning). We need to be partners and collaborators in learning, not providers.  If we do that then, even if we are teaching COBOL, Italian Renaissance poetry, or some other ‘useless’ subject, we will be doing what employers seem to want and need. More importantly, we will be enriching lives, whether or not we make people fiscally richer.

The Myth of 'Learning Styles'

A straightforward journalistic article from the Atlantic that does a decent job of explaining how and why learning styles theories simply don’t work, with a particular focus on VARK. The takeaways are that, yes, people do often prefer to learn in different ways but, no, accommodating their preferences has no effect on comprehension or recall, and, even if it did, it would be doing a disservice to learners to do so because life ain’t like that (with darker implications of teaching people to believe they need something they don’t, thereby actually reducing their capacity to learn). We’ve known this for a really long time. I am still shocked that (at least as recently as 5 years ago) up to 90% of teachers actually believe in the learning styles myth. Of all the people that should know better, teachers are pretty much at the top of the list.

The article points to a good range of recent reliable sources, including:

Another Nail in the Coffin for Learning Styles? Disparities among Undergraduate Anatomy Students’ Study Strategies, Class Performance, and Reported VARK Learning Styles

Learning style, judgements of learning, and learning of verbal and visual information.

The Scientific Status of Learning Styles Theories

Matching Learning Style to Instructional Method: Effects on Comprehension

Proviso: there’s nothing wrong, and everything right, about thinking of different ways to enable people to learn stuff, and learning styles theories all encourage people to do that. As a design tool, that serves as a reminder that there is seldom one best way to teach anything, I’m all in favour of anything that gets the creative juices flowing and that allows learning designers to take and apply different perspectives. This is the sort of thing that increases engagement, interest, and time on task. Even if it is barking mad or positively evil, as long as we don’t let on why we are doing it, any way we find to do this is probably fine. I could, for instance, imagine ways that a ‘men are from Mars, women are from Venus’ perspective could, despite the very unsavoury nonsense behind it, actually result in some diverse approaches to teaching that would benefit everyone, as long as we didn’t try to teach men one way and women another, of course, and as long as we didn’t let on that we had designed it with these thoughts in mind. Substitute whatever demographic divide, bias, bigotry, or preference you like – religion, weight, politics, sexuality, race, drinking habits, liking for cats or dogs, general level of fitness, whatever. As long as you keep it to yourself and only let it affect how you design your teaching, then do what works for you. There’s a slippery slope to be avoided here, and some complexities to be wary of, especially when it changes the content and intended outcomes – if, say, you chose religion as your discriminator, that does not mean you should teach both evolution and intelligent design, though there might be value in remembering that there might be a religious demographic that won’t readily accept any amount of evidence or argument, so you might want to think about how best to help them, just as you should think about how best to help people with any disability.  Let’s just keep it that way, though – a dirty little in-house secret about how we design our teaching by thinking (wrongly or not) about differences between our learners –  and stop inflicting the stupid notion on our students.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/4308469/the-myth-of-learning-styles

E-Learn 2019, Call for Proposals: Due July 8

https://www.aace.org/conf/elearn/call/

Call for Proposals: Due July 8

E-learn has been running since 1996 (originally under the name of WebNet) and is a great conference for researchers into online learning working in higher education and related fields. At its peak, it used to attract about a thousand delegates though this has mostly tailed off a bit in recent years, so it’s a big enough number to ensure diversity and quality, but small enough that you can get to meet many of them. This year the conference is in one of my favourite locations to visit, New Orleans, and it runs from November 4-7, 2019.

What I like most about the conference is its diversity. It typically attracts a great range from more technical to more educationally focused researchers, with a great spread of experience from student researchers to the most famous in the field. There are also usually a lot of other interested and interesting people involved, as this pie chart suggests:

E-learn attendees pie chart

If you want to submit a paper or poster, acceptance rates are about par for the course: it’s certainly not trivially easy to get a paper accepted, but it’s not fiendishly hard.

Disclaimer: I’m on the Executive Committee, have co-chaired it a couple of times, and have only missed about three of the conferences in the last 20 years so I’m obviously a fan!

Originally posted at: https://landing.athabascau.ca/bookmarks/view/4213561/e-learn-2019-call-for-proposals-due-july-8