Obsolescence and decay

Koristka camera  All technologies require an input of energy – to be actively maintained – or they will eventually drift towards entropy. Pyramids turn to sand, unused words die, poems must be reproduced to survive, bicycles rust. Even apparently fixed digital technologies rely on physical substrates and an input of power to be instantiated at all. A more interesting reason for their decay, though, is that virtually no technologies exist in isolation, and virtually all participate in, and/or are participated in by other technologies, whether human-instantiated or mechanical. All are assemblies and all exist in an ecosystem that affects them, and which they affect. If parts of that system change, then the technologies on which they depend may cease to function even though nothing about those technologies has, in itself, altered.

Would a (film) camera for which film is no longer available still be a camera? It seems odd to think of it as anything else. However, it is also a bit odd to think of it as a camera, given that it must be inherent to the definition of a camera that it can take photos. It is not (quite) simply that, in the absence of film, it doesn’t work. A camera that doesn’t take photos because the shutter has jammed or the lens is missing is still a camera: it’s just a broken camera, or an incomplete camera. That’s not so obviously the case here. You could rightly claim that the object was designed to be a camera, thereby making the definition depend on the intent of its manufacturer. The fact that it used to be perfectly functional as a camera reinforces that opinion. Despite the fact that it cannot take pictures, nothing about it – as a self-contained object – has changed. We could therefore simply say it is therefore still a camera, just one that is obsolete, and that obsolescence is just another way that cameras can fail to work. This particular case of obsolescence is so similar to that of the missing lens that it might, however, make more sense to think of it as an instance of exactly the same thing. Indeed someone might one day make a film for it and, being pedantic, it is almost certainly possible to cut up a larger format film and insert it, at which point no one would disagree that it is a camera, so this is a reasonable way to think about it. We can reasonably claim that it is still a camera, but that it is currently incomplete.

Notice what we are doing here, though. In effect, we are supposing that a full description of a camera – ie. a device to take photos – must include its film, or at least some other means of capturing an image, such as a CCD. But, if you agree to that, where do you stop? What if the only film that the camera can take demands processing that is not? What if is is a digital camera that creates images that no software can render? That’s not impossible. Imagine (and someone almost certainly will) a DRM’d format that relies on a subscription model for the software used to display it, and that the company that provides that subscription goes out of business. In some countries, breaking DRM is illegal, so there would be no legal way to view your own pictures if that were the case. It would, effectively, be the same case as that of a camera designed to have no shutter release, which (I would strongly argue) would not be a camera at all because (by design) it cannot take pictures. The bigger point that I am trying to make, though, is that the boundaries that we normally choose when identifying an object as a camera are, in fact, quite fuzzy. It does not feel natural to think of a camera as necessarily including its film, let alone also including the means of processing that film, but it fails to meet a common-sense definition of the term without those features.

A great many – perhaps most – of our technologies have fuzzy boundaries of this nature, and it is possible to come up with countless examples like this. A train made for a track gauge that no longer exists, clothing made in a size that fits no living person, printers for which cartridges are no longer available, cars that fail to meet emissions standards, electrical devices that take batteries that are no longer made, and so on. In each case, the thing we tend to identify as a specific technology no longer does what it should, despite nothing having changed about it, and so it is difficult to maintain that it is the same technology as it was when it was created unless we include in our definition the rest of the assembly that makes it work. One particularly significant field in which this matters a great deal is in computing. The problem occurs in every aspect of computing: disk formats for which no disk drives exist, programs written for operating systems that are no longer available, games made for consoles that cannot be found, and so on. In a modern networked environment, there are so many dependencies all the way down the line that virtually no technology can ever be considered in isolation. The same phenomenon can happen at a specific level too. I am currently struggling to transfer my websites to a different technology because the company providing my server is retiring it. There’s nothing about my sites that has changed, though I am having to make a surprising number of changes just to keep them operational on the new system. Is a website that is not on the web still a website?

Whatever we think about whether it remains the same technology, if it does not do what the most essential definition of that technology claims that it must, then a digital technology that does not adapt eventually dies, even though its physical (digital) form might persist unchanged. This is because its boundaries are not simply its lines of code. This both stems from and leads to fact that technologies tend to evolve to ever greater complexity. It is especially obvious in the case of networked digital technologies, because parts of the multiple overlapping systems in which they must participate are in an ever-shifting flux. Operating systems, standards, protocols, hardware, malware, drivers, network infrastructure, etc can and do stop otherwise-unchanged technologies from working as intended, pretty consistently, all the time. Each technology affects others, and is affected by them. A digital technology that does not adapt eventually dies, even though (just like the camera) its physical (digital) form persists unchanged. It exists only in relation to a world that becomes increasingly complex thanks to the nature of the beast.

All species of technology evolve to become more complex, for many reasons, such as:

  • the adjacent possibles that they open up, inviting elaboration,
  • the fact that we figure out better ways to make them work,
  • the fact that their context of use changes and they must adapt to it,
  • the fact other technologies with which they are assembled adapt and change,
  • the fact that there is an ever-expanding range of counter-technologies needed to address their inevitable ill effects (what Postman described as the Faustian Bargain of technology),  which in turn create a need for further counter-technologies to curb the ill effects of the counter technologies,
  • the layers of changes and fixes we must apply to forestall their drift into entropy.

The same is true of most individual technologies of any complexity, ie. those that consist of many interacting parts and that interact with the world around them. They adapt because they must – internal and external pressures see to that – and, almost always, this involves adding rather than taking away parts of the assembly. This is true of ecosystems and even individual organisms, and the underlying evolutionary dynamic is essentially the same. Interestingly, it is the fundamental dynamic of learning, in the sense of an entity adapting to an environment, which in turn changes that environment, requiring other entities within that environment to adapt in turn, which then demands further adaptation to the ever shifting state of the system around it. This occurs at every scale, and every boundary. Evolution is a ratchet: at any one point different paths might have been taken but, once they have been taken, they provide the foundations for what comes next. This is how massive complexity emerges from simple, random-ish beginnings. Everything builds on everything else, becoming intricately interwoven with the whole. We can view the parts in isolation, but we cannot understand them properly unless we view them in relation to the things that they are connected with.

Amongst other interesting consequences of this dynamic, the more evolved technologies become, the more they tend to be comprised of counter-technologies. Some large and well-evolved technologies – transport systems, education systems, legal systems, universities, computer systems, etc – may consist of hardly anything but counter-technologies, that are so deeply embedded we hardly notice them any more. The parts that actually do the jobs we expect of them are a small fraction of the whole. The complex interlinking between counter-technologies starts to provide foundations on which further technologies build, and often feed back into the evolutionary path, changing the things that they were originally designed to counter, leading to further counter-technologies to cater for those changes. 

To give a massively over-simplified but illustrative example:

Technology: books.

Problem caused: cost.

Counter-technology: lectures.

Problem caused: need to get people in one place at one time.

Counter-technology: timetables.

Problem caused: motivation to attend.

Counter-technology: rewards and punishments.

Problem caused: extrinsic motivation kills intrinsic motivation.

Counter-technology: pedagogies that seek to re-enthuse learners.

Problem caused: education comes to be seen as essential to future employment but how do you know that it has been accomplished?

Counter-technology: exams provide the means to evaluate educational effectiveness.

Problem caused: extrinsic motivation kills intrinsic motivation.

Solution: cheating provides a quicker way to pass exams.

And so on.

I could throw in countless other technologies and counter-technologies that evolved as a result to muddy the picture, including libraries, loan systems, fines, courses, curricula, semesters, printing presses, lecture theatres, desks, blackboards, examinations, credentials, plagiarism tools, anti-plagiarism tools, faculties, universities, teaching colleges, textbooks, teaching unions, online learning, administrative systems, sabbaticals, and much much more. The end result is the hugely complex, ever shifting, ever evolving mess that is our educational systems, and all their dependent technologies and all the technologies on which they depend that we see today. This is a massively complex system of interdependent parts, all of which demand the input of energy and deliberate maintenance to survive. Changing one part shifts others, that in turn shift others, all the way down the line and back again. Some are harder and less flexible than others – and so have more effect on the overall assembly – but all contribute to change.

We have a natural tendency to focus on the immediate, the local, and the things we can affect most easily. Indeed, no one in the entire world can hope to glimpse more than a caricature of the bigger picture and, being a complex system, we cannot hope to predict much beyond the direct effects of what we do, in the context that we do them. This is true at every scale, from teaching a lesson in a classroom to setting educational policies for a nation. The effects of any given educational intervention are inherently unknowable in advance, whatever we can say about average effects. Sorry, educational researchers who think they have a solution – that’s just how it is. Anyone that claims otherwise is a charlatan or a fool. It doesn’t mean that we cannot predict the immediate future (good teachers can be fairly consistently effective), but it does mean that we cannot generalize what they do to achieve it.

One thing that might help us to get out of this mess would be, for every change we make, to think more carefully about what it is a counter-technology for,  and at least to glance at what the counter-technologies we are countering are themselves counter-technologies for. It might just be that some of the problems they solve afford greater opportunities to change than their consequences that we are trying to cope with. We cannot hope to know everything that leads to success – teaching is inherently distributed and inherently determined by its context – but we can examine our practice to find out at least some of the things that lead us to do what we do. It might make more sense to change those things than to adapt what we do to their effects.

 

A simple phishing scam

If you receive an unexpected email from what you might, at first glance, assume to me, especially if it is in atrocious English, don’t reply to it until you have looked very closely at the sender’s email address and have thought very carefully about whether I would (in a million years) ask you for whatever help it wants from you.

Being on sabbatical, my AU inbox has been delightfully uncrowded of late, so I rarely look at it until I’ve got a decent amount of work done most days, and occasionally skip checking it altogether, but a Skype alert from a colleague made me visit it in a hurry a couple of days back. I found a deluge of messages from many of my colleagues in SCIS, mostly telling me my identity had been stolen (it hadn’t), though a few asked if I really needed money, or wanted my groceries to be picked up. This would be a surprising, given that I live about 1000km away from most of them. All had received messages in poorly written English purporting to be from me, and at least a couple of them had replied. One – whose cell number was included in his sig – got a phishing text almost immediately, again claiming to be from me: this was a highly directed and malicious attack.

The three simple tricks that made it somewhat believable were:

  1. the fraudsters had created a (real) Gmail account using the username, jondathabascauca. This is particularly sneaky inasmuch as Gmail allows you to insert arbitrary dots into the name part of your email address, so they turned this into jond.athabasca.ca@gmail.com, which was sufficiently similar to the real thing to fool the unwary.

  2. the crooks simply copied and pasted the first part of my official AU page as a sig, which is pretty odd when you look at it closely because it included a plain text version of the links to different sections on the actual page (they were not very careful, and probably didn’t speak English well enough to notice), but again looks enough like a real sig to fool someone glancing at it quickly in the midst of a busy morning.

  3. they  (apparently) only sent the phishing emails to other people listed on the same departmental bio pages, rightly assuming that all recipients would know me and so would be more likely to respond. The fact that the page still (inaccurately) lists me as school Chair probably probably means I was deliberately singled out.

As far as I know they have not extended the attacks further than to my colleagues in SCIS, but I doubt that this is the end of it. If they do think I am still the Chair of the school, it might occur to them that chairs tend to be known outside their schools too.

This is not identity theft – I have experienced the real thing over the past year and, trust me, it is far more unpleasant than this – and it’s certainly not hacking. It’s just crude impersonation that relies on human fallibility and inattention to detail, that uses nothing but public information from our website to commit good old fashioned fraud. Nonetheless, and though I was not an intended victim, I still feel a bit violated by the whole thing. It’s mostly just my foolish pride – I don’t so much resent the attackers as the fact that some of the recipients jumped to the conclusion that I had been hacked, and that some even thought the emails were from me. If it were a real hack, I’d feel a lot worse in many ways, but at least I’d be able to do something about it to try to fix the problem. All that I can do about this kind of attack is to get someone else to make sure the mail filters filter them out, but that’s just a local workaround, not a solution.

We do have a team at AU that deals with such things (if you have an AU account and are affected, send suspicious emails to phishing@athabascau.ca), so this particular scam should have been stopped in its tracks, but do tell me if you get a weird email from ‘me’.

What is it to be Bayesian? The (pretty simple) math modelling behind a Big Data buzzword | Aeon Videos

This is a great little (16 minute) video that intuitively explains Bayesian probability from a variety of perspectives, but especially in visual (geometric) terms. Very useful for pretty much anyone – this is a critical thinking skill that applies in many contexts – but especially for researchers or programmers struggling with the idea.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/5278878/what-is-it-to-be-bayesian-the-pretty-simple-math-modelling-behind-a-big-data-buzzword-aeon-videos

Is China really the educational powerhouse that the PISA rankings suggest? (tl;dr: not even close)

Administered by the OECD, PISA is basically a set of tests, adapted to each country, that attempt to measure educational performance across a range of skills in order to rank educational systems around the world. The rankings really matter to many countries, and help to determine educational policies across the planet, being especially impactful when countries don’t do well. Often, a low PISA score triggers educational reform (not always ending well), but sometimes countries just stop playing the game. India, for instance, dropped out a decade ago after coming second from the bottom, complaining of lack of adaptation to the Indian context (which is totally fair – India is incredibly diverse, so one measure absolutely does not fit all) though it will be back again next year. There are many reasons to dislike PISA, but the one I want to highlight here is Goodhart’s Law that, when a measure becomes a target, it ceases to be a good measure.

This article – a report on an interview with Andreas Schleicher, OECD Director of Education and Skills (a very smart fellow) – provides some useful food for thought. Though it focuses on China as a case in point, the interview is not so much about China’s ‘success’ as it is about PISA and its limitations in general. Among Schleicher’s more interesting insights is the fact that China’s test results came solely from its four most highly developed and economically successful provinces. These are very unrepresentative of the whole. In fact, China replaced Guangdong in its submission this time round because it was blamed for poorer performance last time, suggesting that the Chinese government’s involvement with PISA is far more concerned with appearing effective on the International stage – on presenting a facade – than on actually improving learning. PISA is a test for countries, and some are quite happy to cheat on the test.

In fact, the biggest contributing factor to test results is, of course and as always, economic. Schleicher notes that, worldwide, the top 10% socioeconomically advantaged students have for at least 10 years consistently outperformed the 10% most disadvantaged students in reading by 141 score points, which equates to approximately three year’s worth of schooling. It is not news that by far the most productive way to improve the effectiveness of educational systems would be to diminish wealth inequalities. It is, though, worth noting that schools play a relatively small role in Chinese education, especially among more prosperous families, with vast amounts of (paid, private) tuition occurring outside schools. Similar extracurricular tuition patterns occur in several of the other highest ranking PISA countries, such as South Korea, Singapore, Japan, Hong Kong, and Taiwan. It is significant that, in these countries, test scores are extremely important in almost every way – economically, culturally, socially, and more – so there is a lot of teaching focused on test results at the expense of almost everything else.

It is also notable – and almost certainly a direct consequence of tests’ importance – that over 80% of Chinese students admit to cheating, which might be more than a minor contributor to the good results. In fairness, cheating rates for the US and Canada are also not too far short of that, correctly implying a serious endemic malaise with our educational systems worldwide (Goodhart’s Law, again), so this is just a relatively slight difference of degree, not of kind. Given the large amount of time spent learning outside school, the high levels of cheating, and the cherry-picking of top performing provinces, the implications are that, far from having a world-leading education system, teaching in China is actually really awful, on average. Among the things that can be gleaned from PISA results are that China performs very badly on productivity (points per hour of learning), and ranks 8th from bottom on life satisfaction for students. It is essentially a failure, by any reasonable measure. The PISA ranking is not quite a fiction but it is close. At least in the case of the high overall placing of China, it certainly fails to correctly measure the effectiveness of the educational system, if results are taken at face value.

There appear to be two distinct patterns among those countries that consistently achieve high PISA results, that appear to divide along broadly cultural lines. The first group includes the likes of China, South Korea, Japan, and Taiwan (all quite notable examples of what Hofsteder describes as collectivist cultures), with high levels of out-of-school tuition, a strong educational emphasis on test scores, and great personal penalties for failure. These countries seem to achieve their high ranking by a very strong focus on passing the tests, with high penalties for failure and great significance for success. As a consequence, their educational systems cannot be seen as standalone causes but, rather, as creators of problems that have to be overcome by other means (most notably in the form of extra-curricular assistance that funds a booming personal tuition economy).  Standard bearers for the other main pattern are Finland and Estonia, as well as Switzerland, and Canada (though the latter two devolve educational responsibility to canton/province, so they are less consistently successful in the rankings). In Hofstede’s terms, these are more individualist societies. In this group, test scores (slightly) tend to be seen as a measure of only one of several consequences of teaching, rather than being the primary motivation for doing it. I am certainly culturally biased, but I cannot help but think this is a better way of going about the process: education is for society, much more than for the individual, and certainly not for economic gain, so it must be understood across many dimensions of value. Whether they agree with me or not, I am almost certain that most educators everywhere would like to think that education is about much more than achieving good test scores. It is only a matter of degree, though. Education in all countries I am aware of relies on extrinsic motivation, and there are large pockets of excellence in the first group and large pockets of awfulness in the second. Averages are a stupid way to evaluate a whole country’s educational system, and they conceal great diversity. The boundaries are also blurred. Estonia, for instance, that is singled out in the article as a success story due to its rapid rise through the rankings, actually also makes extensive use of extra tuition in the form of ‘long day groups’ that take place in schools after curricular instruction. Estonia is no worse than most other countries in this regard, and in some ways superior because such long day groups take the place of at least some of the homework that is widely required in many countries, despite a singular lack of evidence that (on average) it has more than a tiny effect on learning. At least Estonia’s approach involves a modicum of good education theory and evidence to support it.

Overall, I think the main thing that is revealed by the PISA process is that average test scores are, for the most part, an extremely poor means of comparing education systems. Given that it is useful for a government to know how their policies are working, there does need to be some way for them to observe how schools are doing, but it would seem more sensible to rely on trained inspectors reviewing schools, their teaching, the work of children, etc, than on test scores. At the very least they should be considering signs of happiness, motivation, community, and social achievement at least as much as academic achievement. However, Goodhart’s Law would cause its usual harm if such things became the dominant measures of success, and more than the lightest of inspections would normally cause more harm than good. I experienced something not too far removed from this (in the form of OFSTED inspections) in the UK as a parent and school governor back in the 1990s. The results were not pretty. For about a year leading up to them teachers’ workloads were massively strained by the need to report on everything, students suffered, resentments piled up, everyone suffered. Though OFSTED reports did sometimes lead to improvements in particularly bad schools, the effects on the vast majority of schools (and especially on teachers) were disastrous, often radically disrupting work, increasing stress levels beyond reasonable bounds, and leading to more than a few resignations and early retirements from the best, most dedicated teachers who could barely cope with the workloads at the best of times. They were forced to become bureaucrats, which is a role to which teachers tend to be very poorly suited. It was (and, I believe, may still be) beyond stupid, despite best intentions.

What is really needed is something more collegial, that is focused on improvement rather than judgment, that celebrates and builds on success rather than amplifying failure, where everyone involved in the process benefits and no one suffers. The whole point (as far as I understand it) is to improve what we do, not to blame those who fail. Appreciative Inquiry is a good start. Simple things like peer observation (with no penalties, no judgments, just formative commentary) can be more than adequate for the most part at a local level, and are beneficial to both observer and observed. Maybe – if someone thinks it necessary – inspectors (volunteers, perhaps, from the teaching profession) could look at samples of student work from further afield with a similarly positive, formative attitude. It might not provide numbers to compare but, if there were enough of a culture of sharing across the whole sector, and if inspectors came from across the geographical and cultural spectrum, it ought to be good enough to improve practice, and to spread good ideas around, so the intent would be achieved. Governments could receive reports on what actually matters – that things are getting better – rather than on what does not (that things are bad, according to some unreliable measurement that compares nothing of any real value to educators, students, or society). Teaching is a deeply soft technology that cannot be reductively simplified to a relationship of entailment. It can, though, as a lived, creative, social process, be improved. This should be the goal of all teachers, and of all those who can influence the process, including governments. PISA only achieves such results in a tiny minority of extreme cases. For the most part, it actively militates against them because it substitutes education – in all its rich complexity – for test scores. These are not even a passable proxy. They are a gross distortion, an abhomination that can trivially be turned to evil, self-serving purposes without in any way improving learning. Schleicher fully understands this. I wish that the people who his organization serves did too.

Originally posted at: https://landing.athabascau.ca/bookmarks/view/5209267/is-china-really-the-educational-powerhouse-that-the-pisa-rankings-suggest-tldr-not-even-close