An interesting proposal from Horn & Fisher that fills in one of the most gaping holes in conventional quantitative research in education (specifically randomized controlled trials but also less rigorous efforts like A/B testing etc) by explicitly looking at the differences in those that do not fit in the average curve – the ones that do not benefit, or that benefit to an unusual degree, the outliers. As the authors say:

“… the ability to predict what works, for which students, in what circumstances, will be crucial for building effective, personalized-learning environments. The current education research paradigm, however, stops short of offering this predictive power and gets stuck measuring average student and sub-group outcomes and drawing conclusions based on correlations, with little insight into the discrete, particular contexts and causal factors that yield student success or failure. Those observations that do move toward a causal understanding often stop short of helping understand why a given intervention or methodology works in certain circumstances, but not in others.“

I have mixed feelings about this. Yes, this process of iterative refinement is a much better idea than simply looking at improvements in averages (with no clear causal links) and they are entirely right to critique those that use such methods but:

a) I don’t think it will ever succeed in the way it hopes, because every context is significantly different and this is a complex design problem, where even miniscule differences can have huge effects. Learning never repeats twice. Though much improved on what it replaces, it is still trying to make sense through tools of reductive materialism whereas what we are dealing with, and what the authors’ critique implies, is a different kind of problem. Seeking this kind of answer is like seeking the formula for painting a masterpiece. It’s only ever partially (at best) about methodologies and techniques, and it is always possible to invent new ones that change everything.

b) It relies on the assumption that we know exactly what we are looking for: that what we seek to measure is the thing that matters. It might be exactly what is needed for personalized education (where you find better ways to make students behave the way you want them to behave) but exactly the opposite for personal education (where every case is different, where education is seen as changing the whole person in unfathomably rich and complex ways).

That said, I welcome any attempts to stop the absurdity of trying to intervene in ways that benefit the (virtually non-existent) average student and that instead attempt to focus on each student. This is a step in the right direction.

Address of the bookmark: http://www.christenseninstitute.org/publications/a-blueprint-for-breakthroughs/

Why this article is wrong

This article is based on a flawed initial premise: that universities are there to provide skills for the marketplace. From that perspective, as the writer, Jonathan Munk, suggests, there’s a gap between both what universities generally support and what employers generally need, and the perceptions of students and employers about the skills they actually possess. If we assume that the purpose of universities is to churn out market-ready workers, with employer-friendly skills, they are indeed singularly failing and will likely continue to do so.  As Munk rightly notes:

“… universities have no incentive to change; the reward system for professors incentivizes research over students’ career success, and the hundreds of years of institutional tradition will likely inhibit any chance of change. By expecting higher education to take on closing the skills gap, we’re asking an old, comfortable dog to do new tricks. It will not happen.”

Actually quite a lot of us, and even quite a few governments (USA notwithstanding) are pretty keen on the teaching side of things, but Munk’s analysis is substantially correct and, in principle, I’m quite comfortable with that. There are far better, cheaper and faster ways to get most marketable job skills than to follow a university program, and providing such skills is not why we exist. This is not to say that we should not do such things. For pedagogical and pragmatic reasons, I am keen to make it possible for students to gain useful workplace skills from my courses, but it has little to do with the job market. It’s mainly because it makes the job of teaching easier, leads to more motivated students, and keeps me on my toes having to stay in touch with the industry in my particular subject area. Without that, I would not have the enthusiasm needed to build or sustain a learning community, I would be seen as uninterested in the subject, and what I’d teach would be perceived as less relevant, and would thus be less motivating. That’s also why, in principle, combining teaching and research is a great idea, especially in strongly non-vocational subjects that don’t actually have a marketplace. But, if it made more sense to teach computing with a 50 year old language and machine that should be in a museum, I would do so at the drop of a hat. It matters far more to me that students develop the intellectual tools to be effective lifelong learners, develop values and patterns of thinking that are commensurate with both a healthy society and personal happiness, become part of a network of learners in the area, engage with the community/network of practice, and see bigger pictures beyond the current shiny things that attract attention like flames to a moth. This focus on being, rather than specific skills, is good for the student, I hope, but it is mainly good for everyone. Our customer is neither the student nor the employer: it’s our society. If we do our jobs right then we both stabilize and destablize societies, feeding them with people that are equipped to think, to create, to participate, reflectively, critically, and ethically: to make a difference. We also help to feed societies with ideas, theories, models and even the occasional artefact, that make life better and richer for all though, to be honest, I’m not sure we do so in the most cost-effective ways. However, we do provide an open space with freedom to explore things that have no obvious economic value, without the constraints or agendas of the commercial world, nor those of dangerously partisan or ill-informed philanthropists (Zuckerberg, Gates – I’m thinking of you). We are a social good. At least, that’s the plan – most of us don’t quite live up to our own high expectations. But we do try. The article acknowledges this role:

“Colleges and universities in the U.S. were established to provide rich experiences and knowledge to their students to help them contribute to society and improve their social standing.”

Politely ignoring the US-centricity of this claim and its mild inaccuracy, I’d go a bit further: in the olden days, it was also about weeding out the lower achievers and/or, in many countries (the US was again a notable offender), those too poor to get in. Universities were (and most, AU being a noble and rare exception, still are) a filter, that makes the job of recruiters easier by removing the chaff from the wheat before we even get to them, and then again when we give out the credits: that‘s the employment advantage. It’s very seldom (directly) because of our teaching. We’re just big expensive sieves, from that perspective. However, the article goes on to say:

“But in the 1930s, with millions out of work, the perceived role of the university shifted away from cultural perspective to developing specific trades. Over time, going to college began to represent improved career prospects. That perception persists today. A survey from 2015 found the top three reasons people chose to go to college were:

• improved employment opportunities
• make more money
• get a good job”

I’m glad that Munk correctly uses the term ‘perception’, because this is not a good reason to go to a university. The good job is a side-effect, not the purpose, and it is becoming less important with each passing year. Partly this is due to market saturation and degree inflation, partly due to better alternatives becoming more widespread, especially thanks to the Internet. One of the ugliest narratives of modern times is that the student should pay for their education because they will earn more money as a result. Utter nonsense. They will earn more money because they would have earned more money anyway, even if universities had never existed. The whole point of that filtering is that it tends to favour those that are smarter and thus more likely to earn more. In fact, were it not for the use of university qualifications as a pre-filter that would exclude them from a (large but dwindling) number of jobs, they would have earned far more money by going straight into the workforce. I should observe in passing that open universities like AU are not entirely immune from this role. Though not much filtering for ability on entry, AU and other open universities do none-the-less act as filters inasmuch as those that are self-motivated enough to handle the rigours of a distance-taught university program while otherwise engaged, usually while working, are far better candidates for most jobs than those who simply went to a university because that was the natural next step. A very high proportion of our students that make it to the end do so with flying colours, because those that survive are incredibly good survivors. I’ve seen the quality of work that comes out of this place and been able to compare it with that from the best of traditional universities: our students win hands down, almost every time. The only time I have seen anything like as good was in Delhi, where 30 students were selected in a program each year from over 3,000 fully qualified applicants (i.e. those with top grades from their schools). This despite, or perhaps because of, the fact that computing students had to sit an entrance exam that, bizarrely and along with other irrelevances, required them to know about Brownian motion in gases. I have yet to come across a single computing role where such knowledge was needed. Interestingly, they were not required to know about poetry, art, or music, though I have certainly come across computing roles where appreciation of such things would have been of far greater value.

Why this article is right

If it were just about job-ready skills like, in computing, the latest frameworks, languages and systems, the lack of job-readiness would not bother me in the slightest. However, as the article goes on to say, it is not just the ‘technical’ (in the loosest sense) skills that are the problem. The article mentions, as key employer concerns, critical thinking, creativity, and oral and written communication skills. These are things that we should very much be supporting and helping students to develop, however we perceive our other roles. In fact, though the communication stuff is mainly a technical skillset, creativity and problem-solving are pretty much what it is all about so, if students lack these things, we are failing even by our own esoteric criteria.

I do see a tension here, and a systematic error in our teaching. A goodly part of it is down to a misplaced belief that we are teaching stuff, rather than teaching a way of being. A lot of courses focus on a set of teacher-specified outcomes, and on accreditation of those set outcomes, and treat the student as (at best) input for processing or (at worst) a customer for a certificate. When the process is turned into a mechanism for outputting people with certificates, with fixed outcomes and criteria, the process itself loses all value. ‘We become what we behold’ as McLuhan put it: if that’s how we see it, that’s how it will be. This is a vicious circle. Any mechanism that churns students out faster or more efficiently will do. In fact, a lot of discussion and design in our universities is around doing exactly that. For example, the latest trend in personalization (a field, incidentally, that has been around for decades) is largely based on that premise: there is stuff to learn, and personalization will help you to learn it faster, better and cheaper than before. As a useful by-product, it might keep you on target (our target, not yours).  But one thing it will mostly not do is support the development of critical thinking, nor will it support the diversity, freedom and interconnection needed for creative thinking. Furthermore, it is mostly anything but social, so it also reduces capacity to develop those valuable social communication skills. This is not true of all attempts at personalization, but it is true of a lot of them, especially those with most traction. The massive prevalence of cheating is directly attributable to the same incorrect perception: if cheating is the shortest path to the goal (especially if accompanied by a usually-unwarranted confidence in avoiding detection) then of course quite a few people will take it. The trouble is, it’s the wrong goal. Education is a game that is won through playing it well, not through scoring.

The ‘stuff’ has only ever been raw material, a medium and context for the really important ways of being, doing and thinking that universities are mostly about. When the stuff becomes the purpose, the purpose is lost. So, universities are trying and, inevitably, failing to be what employers want, and in the process failing to do what they are actually designed to do in the first place. It strikes me that everyone would be happier if we just tried to get back to doing what we do best. Teaching should be personal, not personalized. Skills should be a path to growth, not to employment. Remembered facts should be the material, not the product. Community should be a reason for teaching, not a means by which it occurs. Universities should be places we learn to be, not places we be to learn. They should be purveyors of value, not of credentials.

Address of the bookmark: http://techcrunch.com/2016/05/08/universities-cant-solve-our-skills-gap-problem-because-they-caused-it/

A nice interview in AUSU’s Voice Magazine – continued at https://www.voicemagazine.org/articles/featuredisplay.php?ART=11372 – with SCIS’s own Maiga Chang, describing his teaching and research. Maiga’s bubbly enthusiasm comes through strongly in this, and his responses are filled with great insights. I particularly like (in the second part of the interview) his thoughts on what makes Athabasca University so distinctive, and its value in the future of learning:

“What are the benefits of teaching at AU compared to traditional universities?
There are differences. They are different from traditional university and AU because we are almost purely online as a university. We teach students with a lot of help from technology. So, in that case, I would say that teaching at AU that we are the pioneers of teaching students with technology, artificial intelligence applications, learning analytics – everything. I would say that this kind of teaching and learning should be the future. As you know, some people start to work on full time jobs after K-12 and some of them go to university for another four years, which means they only learn in traditional classroom or in traditional setting for 12 to 16, maybe 18 years.

How long will you live? How long will you need to learn? You will need to learn for your whole life. When you graduate from high school and university, you cannot go back to university unless you want to quit a job when you want to learn once again. You will need another way of doing life-long learning.

AU gives us the opportunity to create a kind of smart learning environment. So if we can use our research results to make a smarter learning environment, then we can provide students with more personalized learning experiences, which can make them learn more efficient, and learn the things that they really need and want to see on their own way and own pace. That is another good thing for students, I would say, teaching at AU.

What do you think are the strengths of learning at AU?
This is the future. Like the students right now in high school and in primary school, you can ask them. They are trying to use mobile devices to learn. Also, as you know, they will post something on their Facebook or their blog. That is the future. As a parent, around 50% of students at AU have family, even children. When they learn at AU, they are adapting to the future of learning, and, in that case, when their child or children have a question. In my upbringing, I could not ask questions of my parents about using Facebook, but right now, you can, because people use Facebook. Now when you’re taking an AU course, you are sometimes asked to make a video, put it on YouTube, and then you can teach your children, your child.

One more thing is very important. It is self-regulated learning skill. It is very important for everyone because it helps you efficiently learn, or digest, or plan your goal. When you learn with AU, you will learn that kind of skills. You can teach your child and children, and other family members.”

Great stuff! I have one comment to add on a small part of this:  I am firmly with Alfie Kohn and, more recently and in similar vein, Stephen Downes on the side of ‘personal’ rather than ‘personalized’. Personalized learning does have a place in the rich tapestry of tools and methods to help with meeting a range of learning needs, but it is very important that personalization is not something done to learners. Too often, it is the antithesis of self-direction, too often it reinforces and automates teacher control, too often it is isolating and individually focused, too often it sacrifices caring, breadth and serendipity in the service of efficiency, and that efficiency is too often narrowly defined in terms of teacher goals. Knowing Maiga, and seeing what else he talks about in this interview, I’m pretty sure that’s not what he means here! Personal learning means focusing on what learners need, want, find exciting, interesting, challenging, problematic or mind-expanding. It is inherently and deeply a social activity supported by and engaged with others, and it is, at the same time, inherently a celebration of diversity and individuality. For some skills – mechanical foundations for example, or as controllable advisory input – personalization can contribute to that, but it should never usurp the personal.

Address of the bookmark: https://www.voicemagazine.org/archives/articledisplay.php?ART=11338&issue=2414

This is a nice set of reflections on the potential value of GitHub to teachers. The title is broader, referring to the Git source code version control system, an open standard with hundreds of implementations, but most of the article is about GitHub, a closed commercial system that packages up Git with a deeply social workflow and friendly interface, making the bulk of its money from those that want support for closed projects and teams rather than open source goodness. Ben rightly points out that a source control system is great for text but less great for binary files and that, despite the quite friendly interface, there is quite a learning curve needed to use it effectively, especially if you are not used to the complexities of writing MarkDown code. Essentially, though it is a soft tool that can be repurposed and reassembled in many different ways, it is built for programmers, and structured in ways that support application development, not other things.

Ben’s suggestions are (typically) thought-provoking and good…

1. An open source, freely-available content control tool designed for people working with non-code. It’s okay for it to not know about fancy file formats like Word, but it should be able to handle more than line-by-line changes. Perhaps call it scribe.
2. A proprietary, beautiful city ecosystem built around it. A ScribeHub.

Nice idea and, as he observes, one that some people have already tried and failed to do, providing some good examples of tools that go part of the way. There’s a good discussion of some of the issues of doing so in the follow-up comments to a post by David Wiley a while back. That said, the big advantage of GitHub is that it does already exist (and is thriving) and does get used for much more than just coding. I really like some of the innovative uses of GitHub for things like journal production: https://github.com/ReScience/ for instance, uses it to make articles and research into living documents, updated as reviews and replications come in. But, as Ben says, it is not optimal for anything other than coding and text documentation and, though there are some great exemplars, it is not likely to hit the mainstream as an alternative means of production outside the coding and documentation community for some time, if ever. Also, much as I love GitHub for its innovative and smooth community integration, it is a commercial monolith. Such things should be distributed and open.

What makes GitHub so cool

Perhaps the biggest differentiating feature in GitHub that makes it stand out from other similar tools is the combination of (for the unpaid variant) required openness, and the ability for anyone at all to make a ‘pull request’. Anyone can make a copy (a ‘fork’) of an existing GitHub project and (and here is the good bit), if they make changes that would be useful in the upstream project, submit a pull request to the author(s) so that their changes can be reincorporated (merged) back into the main branch of the code. Github provides tools that, at least for text, make such merging relatively pain-free. Through this mechanism, the work of many loosely coupled people can cooperatively work on complex projects without the need for further mechanisms of collaboration, teams, collaboration, or complex project management. GitHub does, of course, have rich communication tools for discussing such changes and passing them back and forth, so it can be used very effectively for closed teams as well as in a more open, networked community, but its central social motif is the network, not the group.

An idea

I have been thinking for some time about building a programming course that uses GitHub or, perhaps better, an open variant such as GitLab, or a related coding support tool with similar intent like Phabricator. The basic idea would be that the course itself would evolve through pull requests – if students or others have ideas for improvements, they would simply implement them and submit a pull request which the course owner could choose to merge or reject. Others could, of course, build their own versions of the course at will. I don’t think this is particularly original in itself – many have built OERs this way – but it makes sense to me as both a way of actually hosting a course, and as a way of building in student participation in the development and evolution of a course. Amongst other things, it opens up the potential for students to customize courses for their particular needs: if the basic model contains stuff that is irrelevant or already known, they could adapt it to the way they want it and, of course, share that with others. This in turn opens up some interesting options for scalability and personalization (the good sort). Rather than providing a single, monolithic MOOC, courses could branch off into many related versions, each with its own communities and interests. Someone might, for example, adapt the structure for a different language, focus down on a particular element, restructure it for different pedagogical designs, or extend it for more or less advanced learners. As the ‘course’ itself would be hosted on GitHub (or whatever) there would be no need for additional tools, and the course communities/cohorts could relatively easily blend with one another, or overlap. There would be evolutionary competition between the various branches, perhaps, leading to ever better (or, more accurately, better adapted) versions of the courses.

At least a part of the assessment of the course would be based around taking an existing codebase (in some possible variants, perhaps the code used for the system employed to host the course?) and making improvements to it. Credit would be awarded to those whose pull requests were accepted. One particularly nice thing about that is that all work would, by its very nature, be original. There would be no value at all in simply copying what someone else had done, and success would be measured according to real-world metrics: it would have to be good enough to enter production. It might get a bit complicated as the course matured and there were fewer obvious things to be improved, but I have yet to come across any perfect software beyond very trivial and, with a plugin/service-based architecture the potential for improvements could be virtually limitless. There’s scope for most skill levels, apart from absolute beginners, here. And even relative newbies could contribute to things like documentation.

The idea appeals to me though, as others have found when trying to do something similar with OERs, the complexities mount up pretty quickly. One of the issues is that, unlike in the case of most programming code, one size does not fit all: it is not about producing one useful course or toolset. We are not talking about building an open textbook here, but a course that is suitable for many people in many different contexts. It is therefore more likely that forks would be more useful than merges for most people. In the coding community, this can be a problem – you wind up with many similar forks of code, each of which goes its own (increasingly incompatible) way, diluting the value and community around the original and making it difficult to choose between them (for instance, the many forks of MySQL or the two major branches of Open/LibreOffice). Big products can spawn so many forks and pull-requests that the original authors can be overwhelmed. For courses, forking would allow for the kind of repurposing – contextualization around individuals and communities – that makes OERs worthwhile in the first place. More than with open source applications, though, there would also be issues with diluting the learning community: this might be a benefit for something like a MOOC, where numbers are too large to be managed in the first place, but not so good for smaller courses.  There’s a balance to be sought. Having recently tried (and I am still trying) to incorporate changes from a main branch into a modified version of an OER course, I can verify that it can be fiendishly complex.  I want to maintain our own localizations but the updates are great and, in some cases, necessary. Merging is really difficult, because there is a great deal more involved than simple text, hierarchical directories and a few dependencies to deal with.

A system that would do as Ben suggests for complex media would be a great help in such things. Among those rich media I would love it if it could cope with, say, an exported Moodle course, where it is not just content but process and structure that needs to be tracked, and where changes to structure could greatly impact the meaning and value of the content. The complex, soft dependencies and need for narrative flow make such things structurally very different from the relatively proscribed ways that programs can change, so I don’t have a clear idea of how that could be done right now. It would certainly be possible to use an XML interchange format to track such things but those are made for machines, not people, to read. In fact, the only human-friendly way that I can think of for dealing with it would be to build it into the authoring environment itself – to have a Git-like thing at the backend of (say) Moodle. At Athabasca we do kind-of the same sort of thing using Alfresco to track changes, but the process is clunky, discontinuous, lacks the elegant cooperation of GitHub, is very document-centric (no fine-grained merging at all), and is very much a group, not a network environment, with teams and roles that are anything but open and that exist in very rigid organizational hierarchies, with roles that limit what they can do, and only a single, unforked course as the outcome.

Perhaps such a project – to build that friendlier front end – might be what course takers might use as raw material. Early, and more advanced, takers of the course would be building the infrastructure for later students. I rather like the idea.

Address of the bookmark: https://medium.com/@benwerd/git-for-teachers-e993d2ca423d#.nqby85xqs