learner-teaching-learning analytics

I’ve been having some interesting discussions in Banff this week with folks interested in ‘learning analytics’. I put it in quotes because I’m not convinced that it is a) a distinct field or b) one thing.

Ignoring issues of massive overlaps and shared values with other fields (such as data mining, collaborative filtering, adaptive hypermedia, natural language processing, learning design and evaluation and so on) which make it hard to distinguish at times, it seems to me that there are at least three subfields:

learner analytics: used by admins, policy makers, governments and so on to see what learners are doing with a view to taking some action at a pragmatic or policy level as a result. May also be used by teachers to monitor and understand learners and their needs. Rarely, but potentially, of use to learners.

teaching analytics: looking at the success or otherwise of teaching interventions – courses, assessments, teaching acts, content construction, learning design, etc, with a view to changing the teaching process to make it better. Pretty much exclusively the domain of those involved in the teaching process like teachers and instructional designers.

learning analytics: looking at how people are learning, including construction of artefacts, interactions with others, progression, etc, with a view to taking direct action to improve it, usually (but by no means necessarily) by and for the learner.

I care about learning analytics and see great practical value in teaching analytics. Analysing learning and teaching is almost entirely about helping people to learn and, while it may be poorly done, the intentions are almost all aimed at making learners’ lives better. Analysing learners involves some murkier areas: it may have many motivations, including potentially risky ones like implementing efficiencies, targeting for marketing, allocating resources and so on as well as clearly good things like identifying under-represented groups or at-risk learners. I suspect that it may become the most popular analytics domain in education but, because of the dangers, it demands more serious cross-disciplinary and ethically well-considered research than the others. 

Survey: 85% of Employees Under 25 Use Personal E-Mail Accounts for Work

This accords with my experience – in fact, I’m amazed that the figure isn’t higher across the board.

Email is the boring plumbing of the Internet but so totally essential it seems remarkable that more effort is not put in to supporting it properly so that such problems do not arise. I don’t mean management of the software and hardware, removal of spam, virus protection etc- that part I take as a given. I mean the real management, in which managers listen to what the people using their machines need, proactively anticipate their wants, monitor how they are doing. A free email account from Google can run rings round what most organisations can provide, in reliability, security, spam protections, capacity, speed, accounting, manageability. If corporate email could serve needs better then people would be less inclined to bypass it, but the investment needed if networking departments were to listen and act on what people want and need would be huge, given decades of underinvestment. Necessary, though. 

 

Address of the bookmark: http://www.readwriteweb.com/enterprise/2011/02/survey-85-of-employees-under-2.php?utm_source=pulsenews&utm_medium=referral&utm_campaign=Feed%3A+readwriteweb+%28ReadWriteWeb%29

The Institutions of Theseus

From Wikipedia…

The ship wherein Theseus and the youth of Athens returned [from Crete] had thirty oars, and was preserved by the Athenians down even to the time of Demetrius Phalereus, for they took away the old planks as they decayed, putting in new and stronger timber in their place, insomuch that this ship became a standing example among the philosophers, for the logical question of things that grow; one side holding that the ship remained the same, and the other contending that it was not the same.

—Plutarch, Theseus (http://en.wikipedia.org/wiki/Ship_of_Theseus)

Over the past few days I have been in Japan thanks to the OU of Japan, talking a bit but mainly listening to the great and good talking about how their interest groups (universities, publishers and libraries) should react to a world of increasingly rapid and disruptive change. What interests me greatly about all of this (apart from the research, analyses and arguments presented, of course, much of which has been wonderful) is the assumption, by all, that their particular institutional sector should persist, no matter how much it may change. Libraries become learning centres, universities become publishers, publishers become universities… but the fundamental unit and mindset in each endangered institution persists. We imagine many interesting new futures for our institutions but the end of the institution remains unimaginable. 

I wonder, at what point would we stand up and say ‘we’ve had a good run and we did a good job in our time, but now we are irrelevant and getting in the way of making life better. It’s time to call it a day. What shall we do next?’ I don’t think most of us would ever do this. We just have too much investment in what we have done and it is too much trouble to change the whole thing at once. It’s a bit like realizing that your house is not very well designed or efficient and therefore knocking it down and building a new one. But, though we may resist that for decades or even centuries, at some point, it has to happen. The trouble with the incessant expansion of the adjacent possible is that the argument for demolition gets stronger at an accelerating rate.

We have learned to accept this with computers for some time. From the late 1980s to the early 2000s I had one computer, albeit one very much like the Ship of Theseus, which had no parts in it’s final incarnation that were present in the original of the late 1980s. But since then, while some information has stayed on my machines since the mid 1980s, when one machine no longer suits my needs, I get another. It’s cheaper and simpler than modifying an old one. The same has been true of cellphones for as long as they have been with us.

It is, of course, a lot lot harder to do that with big institutions and businesses. Hundreds or even thousands of years of slow modification and adaptation have made things like libraries, universities and publishers a very fixed and deeply interwoven part of culture and society, not to mention infrastructure. It’s not as simple as replacing a computer to replace an education system on which almost every other institution and industry in some way depends, especially once we move beyond higher education to schools. We can keep modifying and replacing parts for a long time, I guess, but at some point we will need a new ship.

Interestingly, the analogy holds in the extended Ship of Theseus paradox, in which the rotten planks of the original are collected and eventually reassembled to form the original ship while the ship that bears the name no longer contains any physical parts from the original (which is the ‘real’ ship? At what point does one change to the other?) – libraries are starting to pick up the discarded bits that schools used to own, publishers are taking over various roles of universities, universities are taking pieces of community libraries and trying to become publishers…the list goes on. But maybe we could build better ships instead?

When radical change to universities happens it will probably happen quite fast and it will probably slip in rather unexpectedly from somewhere else entirely. Erik Duval made a great provocative throwaway comment in a panel with assembled publishers that Wikipedia had made other encyclopedias irrelevant already. No one argued that one, but a few looked uncomfortable. Some tried to suggest a hybrid of open and closed content as the best way forward but they were unconvincing. Thanks to the Web, libraries too are already finding their traditional role of custodians of information largely usurped and are becoming something almost entirely unrecognisable to librarians of the past – far closer to educators and infrastructure providers than librarians in many cases. Education, however, is far more deeply intertwined with other things, from work and family patterns to accreditation, from relatively unencumbered knowledge generation to the preservation of culture. And it comes with quasi-religious trappings. It’s a tougher nut to crack.

The separation of accreditation and learning support may be the catalyst for a cascade of change. PLAR/APEL and challenge for credit processes already largely separate learning from accreditation and may be the means to achieve a disruptive and positive evolution that is more widespread in its effects. Accreditation is the biggest single link between educational institutions and the rest of the social ecology so, when that goes, the rest becomes more open to change, competition and evolutionary pressure.

This is a risky process – high quality methods of accreditation are vital if we are to see positive change: if the separation leads to more standardised exams, for example, then the result would be a disaster for learning – teaching the most efficient way to pass tests will trump deeper learning every time and we could expect to see education mills tuning themselves to fit the tests in the worst way possible.

Competence-based methods that valorise diversity and creativity, like portfolios, are vital if we are to see a positive revolution and not make life worse than it is already. Richer competence-based assessment is also essential if we are to preserve the value of expertise in professors and lecturers: if standard tests become the norm, assessor roles will be significantly deskilled and the diversity that is one of the great parts of the university ship that must be preserved and nurtured will be largely stamped out. It’s not that there is no place for that kind of test but it should not become the norm. It is important that, in separating learning and assessment, both should remain useful roles for a university, even if other companies, networks and organisations may provide one or the other parts for the learners. There should be nothing to prevent learners from choosing to have both provided by a university if they want, and it might be good for them to do so in many cases. But they should also be able to choose to learn elsewhere and be assessed by the university, or learn at the university and be assessed elsewhere, or bypass the university altogether, if they wish.

If we can cut the direct rope linking higher education and promotion/job finding, then the stage will be set for potential radical and positive change which will, peculiarly, be a return to traditional values in which universities are again set firmly in their role as creators and nurturers of knowledge.

I’m quite looking forward to that.

 

Why LMSs Aren’t the Answer « Educational Technology and Change Journal

I’ve just (this morning) given a talk making almost exactly the points mentioned here so, unsurprisingly, I like the points being made here. The LMS makes things easier by reducing choice – the result is a tendency to limited creativity and inspiration. Assembling tools using a pick and choose or mix and mash approach gives back the control, to some extent though, in this article, I think it leaves more softness in the technology than many would find easy or comfortable. Achieving the right level of softness and hardness in a given context is a really important challenge (and assembly is the right direction to find an answer)

Address of the bookmark: http://etcjournal.com/2011/02/22/why-lmss-arent-the-answer/

Four hours on your computer doubles risk of heart disease

Not just heart disease, but pretty much any way to die. Doing exercise after sitting for 4 hours doesn’t help one bit. Some serious implications for online learners but it is actually about as bad if you just spend time writing at a desk or sitting reading a book. It’s not the computers per se – it’s the fact that people spend too long sitting down in one position and not moving. The recommendation is to get up and move around every 20 minutes but it strikes me that we should probably avoid sitting in one position for a long time as a matter of course. Tablet computers, easily portable notebooks and alternative interfaces (eg Wiimote, voice, etc) seem a potential solution (if we want them to be and make a point of it) as they tend to encourage variety in posture and location.

Address of the bookmark: http://blogs.forbes.com/eco-nomics/2011/01/12/four-hours-on-your-computer-doubles-risk-of-heart-disease/

Test-Taking Cements Knowledge Better Than Studying, Researchers Say – NYTimes.com

An interesting article (thanks to Mary Pringle for alerting me to this).

The claimed finding is that fact retrieval is improved through taking tests. Or, to put it another way that better reflects what is actually being researched here, taking tests improves the ability of students to take similar tests. Hmm. That’s news?

Any sensible pedagogical design will include something like tests (maybe not with that name) as an integral part of the learning process. It is an essential part of the metacognitive process and fits well with work on learning cycles by Lewin, Kolb and others over the past hundred years or so, and aligns perfectly with a constructivist view of knowledge. We need opportunities to connect new knowledge with old and to apply it. It’s not the end of the process but it’s an important step along the way. Testing forces us to confront our beliefs, reflect on our knowledge, apply it, think twice about what we know and don’t know, identify the flaws, take remedial action, and to do it in a (typically) ‘safe’ context before we have to apply it for real. 

Like almost all such articles where people attempt to ‘experiment’ with different approaches to education, there appear from what is reported in NYT to be at least a couple of gaping methodological flaws:

  1. The amount of time spent on task seems to have been largely ignored. By my reckoning, the control group in the first experiment (reading only) spent 5 minutes on the task, the repeat-reading group spent about 20 minutes, the concept mapping group spent an unspecified amount of time (probably extended because of the extra cognitive load involved in the diagramming process, so time actually thinking about what was being learned was not that great) and the test group spent at least half an hour, all of it relating to the content to be learned. The conclusion that might be drawn from this is that the longer one spends thinking about something one has learned, the better one will have learnt it. Indeed, looking at the results and bearing this in mind, it is surprising that the control groups did not do worse than they did.
  2. To make it even less reliable, it appears that no account has been taken of the fact that those using tests were actually practicing the very skills needed to do better on tests – exam technique can be learned just like any other skill.
  3. It is not clear whether or not feedback was given on the results of the test. If it were, the simple fact that caring was shown by whatever or whoever gave the feedback would have had a notable effect on learning. Even if not, tests would have highlighted to the learners what they did and didn’t know more effectively than concept maps or reading – that’s why we give learners opportunities to practice applying their knowledge.

What this study (on the face of it, it’s not an experiment because of the lack of proper control for highly significant variables) does suggest is that the simple application of concept mapping does not greatly improve fact learning as a matter of course. This is obvious. We know that tools do not improve learning: it is not the tools themselves but how they are used that turns them into a learning technology. The devil is in the detail: how much preparation was provided for those using concept maps? How much time and effort was spent in the mechanical process of map construction relative to time spent in reflecting on what had been learned? Were users of concept maps provided with sufficient training to allow them to use the tools to identify gaps in knowledge as well as connections? Were they able to get feedback or share maps with others?

Perhaps I am being unfair to the researchers and I’m looking forward to seeing the real article to find out more about how the study was conducted – sadly, Science does not make articles published online available as part of AU’s access package so we have to wait till it appears in the journal itself before we can read it.

Address of the bookmark: http://www.nytimes.com/2011/01/21/science/21memory.html?ref=science

Multitasking: The Brain Seeks Novelty

This is your brain on Twitter/facebook/email/iPhone/TV/crack cocaine. Very simple and what should be an obvious message for site designers in online learning: never show exactly the same page twice. In learning we move from one novelty to the next – that’s part of what makes it such fun. One problem with a typical coursel site that shows a static content hierarchy (rather a lot of LMS – based courses) is that the jump-off page tends to vary little. In a program I used to run I insisted that the entry point for every course was the discussion forum, driven by Michael Moore’s transactional distance theory more than anything else. However, it was a pretty good motivator too, bringing people to the sites more frequently than those for static content course sites. I thought it was just because people like to socialise and also had to keep visiting in order to know what was happening. Turns out that dopamine may have played a role in this too!

Address of the bookmark: http://www.huffingtonpost.com/russell-poldrack/multitasking-the-brain-se_b_334674.html

The Educational Benefit of Ugly Fonts

Disfluency turns out to be rather good: people seem to learn better from things that are harder to read. Time to rethink those strategies about always making things as easy and clear as possible!

Address of the bookmark: http://www.wired.com/wiredscience/2011/01/the-benefit-of-ugly-fonts/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29

Internet Explorer falls behind Firefox in Europe for first time

Why did it take so long? Internet Exploder is about passable nowadays but not even close to best of breed and the extreme lack of trust in it that Microsoft meticulously cultivated over a decade or more should have put it to bed much sooner. Despite well publicised recent successes such as its ability to trap more malevolent sites than the rest (necessary, one might argue, because it is notably more vulnerable when it reaches them) and some long-needed if half-hearted improvements in security, speed and standards-compliance IE remains, at best, a mediocre alternative to Firefox, Chrome, Safari or Opera. It is interesting, therefore, how a technology that is manifestly inferior to virtually all of its competitors should still occupy one of the top two places in the charts. Force of history (the big one) driven by preferential attachment and the Matthew Principle, spawned by uncompetitive practices and aggressive marketing together with some very unwise choices by Netscape at precisely the wrong time in its history might help to explain a lot of it but, given the trouble IE has caused and the ease with which it can be replaced,  I can’t help feeling there is another dynamic at work.

Pleasingly, Landing visitors seem to know better and we have seen far more Firefox users on the site than any other browser by a very sizeable margin. Indeed, even Safari seems to give IE a run for its money and Chrome, while still very much a minority browser, is starting to show up on the charts as more than just a flat line near the bottom of the Y axis.

Address of the bookmark: http://www.guardian.co.uk/technology/blog/2011/jan/04/internet-explorer-falls-behind-firefox-europe

Year-End Stats from MIT Point to Increasing Popularity of Open Educational Resources

Big numbers. Very big numbers. Open courseware has been wonderful for MIT and wonderful for learners around the world. The costs of infrastructure to support this may be high but the payoffs seem more than worth it.

Address of the bookmark: http://www.readwriteweb.com/archives/year-end_stats_from_mit_point_to_increasing_popula.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+readwriteweb+%28ReadWriteWeb%29