Every attempt to manage academia makes it worse

Excellent post from Mike Taylor on the inevitable consequences of the use of incentives to shape a system (in this case, an educational system). As Mike notes, the problem is well-known and well understood, yet  otherwise intelligent people continue to rely on extrinsic incentives to attempt to shape behaviour. It’s a classic Monkey’s Paw problem – you get what you wish for but something very bad will inevitably happen, often worse than the problem you are trying to solve. We can make people do things with extrinsic incentives (reward and punishment), sure, but in doing so we change the focus from what we want to achieve to the reward itself, which invariably destroys intrinsic motivation to do what we want done, reinforces our power (and thus the weakness of those we ‘incentivize’), and ultimately backfires on us in tragically predictable ways, because what we actually want done is almost never the thing we choose to measure.

some consequences of incentives, Edwards and Roy (2017)

Our educational systems (and many others) are built around extrinsic incentives, from grades through to performance-related pay through to misguided research assessment exercises, evaluations based on publication records, etc. The consequences are uniformly dire.

Mike quotes Tim Harford (from http://timharford.com/2016/09/4035/) as providing what seems to me to be the only sensible solution:

“The basic principle for any incentive scheme is this: can you measure everything that matters? If you can’t, then high-powered financial incentives will simply produce short-sightedness, narrow-mindedness or outright fraud. If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability.”

Well said. Except that I would add that the effects on motivation of any incentive scheme are always awful, and that’s the biggest reason not to do it. It’s not just that it doesn’t achieve the results we hope for: it’s that it is unkind and dehumanizing. With that in mind, I wouldn’t tell them to do the job to the best of their ability. I might ask them. I might help to structure a system so that they and everyone else can see the positive and negative consequences of actions they take. I might try to nurture a community where people value one another and are mutually supportive. I might talk to them about what they are doing and offer my support in helping them to do it better. I might try to structure the system around what people want to do rather than trying to make them fit in the system I want to build. At least, that’s what I would do on a good day. On a bad day, under pressure from multiple quarters, overworked and overstressed, I might fall back on a three line whip or a plea to do their bit. I might make trades (‘do this and I will take away that’) or appeal to a higher authority (‘the Dean says we must…’) or to my own authority (‘this has to be done and you are the best one to do it..’), or to duty (‘it is in our contract that we have to do performance assessments…’).  And that’s where the problems begin.

Mike recommends Tim Harford’s ‘The Undercover Economist’ as a way out of this loop. I will read this, as I have read many books offering similar insights. It seems at first glance to fit very well with the findings of self-determination theory as well as behavioural economics. However, though the causes described here are the result of a failure to understand human motivation, this is, at heart, a systems problem of a broader nature: I recommend The Systems Bible (formerly Systemantics) by John Gall Systemantics by John Gall (formerly the Systems Bible) for a comprehensive set of explanations of the kinds of phenomena that give rise to stupid behaviour by groups of intelligent people. The book is deliberately funny, but the underlying theory on which it is based is extremely sound.

Address of the bookmark: https://svpow.com/2017/03/17/every-attempt-to-manage-academia-makes-it-worse/

Intrinsic and Extrinsic Motivation

A short article from Lisa Legault that summarizes self-determination theory (SDT) and its findings very succinctly and clearly. It’s especially effective at highlighting the way the spectrum of extrinsic-to-intrinsic motivation works (including the integrated/identified/introjected continuum), and in describing the relationships between autonomy, competence, and relatedness. Nothing new here, nothing inspirational, just a useful resource to point people at so they can learn about the central tenets of SDT

Address of the bookmark: https://www.researchgate.net/profile/Lisa_Legault/publication/311692691_Intrinsic_and_Extrinsic_Motivation/links/5856e60d08ae77ec37094289.pdf

Alfie Kohn: "It’s bad news if students are motivated to get A’s" – YouTube

A nice one-minute summary of Alfie Kohn’s case against grades at www.youtube.com/watch?v=EQt-ZI58wpw

There’s a great deal more Kohn has to say on the subject that is worth reading, such as at http://www.alfiekohn.org/article/case-grades/ or http://www.alfiekohn.org/article/grading/ or an interview at http://www.education.com/magazine/article/Grades_Any_Good/

From that interview, this captures the essence of the case pretty well:

“The research suggests three consistent effects of giving students grades – or leading them to focus on what grade they’ll get. First, their interest in the learning itself is diminished. Second, they come to prefer easier tasks – not because they’re lazy, but because they’re rational. After all, if the point is to get an A, your odds are better if you avoid taking intellectual risks. Third, students tend to think in a more superficial fashion – and to forget what they learned more quickly – when grades are involved.

To put it positively, students who are lucky enough to be in schools (or classrooms) where they don’t get letter or number grades are more likely to want to continue exploring whatever they’re learning, more likely to want to challenge themselves, and more likely to think deeply. The evidence on all of these effects is very clear, and it seems to apply to students of all ages.

As far as I can tell, there are absolutely no benefits of giving grades to balance against these three powerful negative consequences – except that doing so is familiar to us and doesn’t take much effort.”


Note: if this video shows up as a blank space in your browser, then your security settings are preventing embedding of untrusted content in a trusted page. This video is totally trustworthy, so look for the alert to override it, typically near the address bar in your browser.

Address of the bookmark:

The cost of admission to the unlearning zone

picture of dull classroom (pubic domain)I describe some of what I do as ‘unteaching’, so I find this highly critical article by Miss Smith – The Unlearning Zone –  interesting. Miss Smith dislikes the terms ‘ unteaching’ and ‘unlearning’ for some well-expressed aesthetic and practical reasons: as she puts it, they are terms “that would not be out of place in a particularly self-satisfied piece of poststructuralist literary analysis circa 1994.”  I partially agree. However, she also seems equally unenamoured with what she thinks they stand for. I disagree with her profoundly on this so, as she claims to be new to these terms, here is my attempt to explain a little about what I mean by them and why I think they are a useful part of the educators’ lexicon, and why they are crucially important for learners’ development in general.

First the terms…

Yes, ‘unteaching’ is an ugly neoligism and it doesn’t really make sense: that’s part of the appeal of using it – a bit of cognitive dissonance can be useful for drawing attention to something. However, it is totally true that someone who is untaught is just someone who has not (yet) been taught, so ‘unteaching’, seen through that light, is at best pointless, at worst self-contradictory.  On the other hand, it does seem to follow pretty naturally from ‘unlearning’ which, contrary to Miss Smith’s assertion, has been in common use for centuries and makes perfect sense. Have you ever had to unlearn bad habits? Me too.

As I understand it, ‘unteach’ is to ‘teach’ as ‘undo’ is to ‘do’.  Unteaching is still teaching, just as undoing is still doing, and unlearning is still learning. Perhaps deteaching would be a better term. Whatever we choose to call it, unteaching is concerned with intentionally dismantling the taught belief that teaching is about exerting power over learners to teach, and replacing it with the attitude that teachers are there to empower learners to learn. This is not a particularly radical idea. It is what all teachers should do anyway, I reckon. But it is worth drawing attention to it as a distinct activity because it runs counter to the tide, and the problem it addresses is virtually ubiquitous in education up to, and sometimes at, doctoral level.

Traditional teaching of the sort Miss Smith seems to defend in her critique does a lot more than teach a subject, skill, or way of thinking. It teaches that learning is a chore that is not valuable in and of itself, that learners must be forced to do it for some other purpose, often someone else’s purpose. It teaches that teaching is something done to students by a teacher: at its worst, it teaches that teaching is telling; at best, that teaching involves telling someone to do something. It’s not that (many) teachers deliberately seek these outcomes, but that they are the most likely lessons to be learned, because they are the ones that are repeated most often. The need for unteaching arises because traditional teaching, with luck in addition to whatever it intends to teach, teaches some terrible lessons about learning and the role of teaching in that process that must be unlearned.

What is unteaching?

Miss Smith claims that unteaching means “open plan classes, unstructured lessons and bean bags.” That’s not the way I see it at all. Unlike traditional teaching, with its timetables, lesson plans, learning objectives, and uniform tests, unteaching does not have its own technologies and methods, though it does, for sure, tend to be a precursor to connectivist, social constructivist, constructionist, and other more learner-centred ways of thinking about the learning process, which may sometimes be used as part of the process of unteaching itself. Such methods, models, and attitudes emerge fairly naturally when you stop forcing people to do your bidding. However, they are just as capable of being used in a controlling way as the worst of instructivist methods: the number of reports on such interventions that include words like ‘students must…’, ‘I make my students…’ or (less blatantly) ‘students (do X)’ far outnumber all others, and that is the very opposite of unteaching. The specific technologies (including pedagogies as much as open-plan classrooms and beanbags) are not the point. Lectures, drill-and-practice and other instructivist methods are absolutely fine, as long as:

  1. they at least attempt to do the job that students want or need,
  2. they are willingly and deliberately chosen by students,
  3. students are well-informed enough to make those choices, and
  4. students can choose to learn otherwise at any time.

No matter how cool and groovy your problem-based, inquiry-based, active methods might be, if they are imposed on students (especially with the use of threats for non-compliance and rewards for compliance – e.g. qualifications, grades, etc) then it is not unteaching at all: it’s just another way of doing the same kind of teaching that caused the problem in the first place. But if students have control – and ‘control’ includes being able to delegate control to someone else who can scaffold, advise, assist, instruct, direct, and help them when needed, as well as being able to take it back whenever they wish – then such methods can be very useful. So can lectures. To all those educational researchers that object to lectures, I ask whether they have ever found them valuable in a conference (and , if not, why did they go to a conference in the first place?). It’s not the pedagogy of lectures that is at fault. It’s the requirement to attend them and the accompanying expectation that people are going to learn what you are teaching as a result. That’s, simply put, empirically wrong. It doesn’t mean that lecturees learn nothing. Far from it. But what you teach and what they learn are different kinds of animal.

Problems with unteaching

It’s really easy to be a bad unteacher – I think that is what Miss Smith is railing against, and it’s a fair criticism. I’m often pretty bad at it myself, though I have had a few successes along the way too. Unteaching and, especially, the pedagogies that result from having done unteaching, are far more likely to go wrong, and they take a lot more emotional, intellectual, and social effort than traditional teaching because they don’t come pre-assembled. They have no convenient structures and processes in place to do the teaching for you.  Traditional teaching ‘works’ even when it doesn’t. If you throw someone into a school system, with all its attendant rewards, punishments, timetables, rules and curricula, and if you give them the odd textbook and assessment along the way, then most students will wind up learning something like what is intended to be taught by the system, no matter how awful the teachers might be. In such a system, students will rarely learn well, rarely persistently, rarely passionately, seldom kindly, and the love of learning will have been squashed out of many of them along the way (survivors often become academics and teachers themselves). But they will mostly pass tests at the end of it. With a bit of luck many might even have gained a bit of useful knowledge or skill, albeit that much will be not just wasted and forgotten as easily as a hotel room number when your stay is over, but actively disliked by the end of it. And, of course, they will have learned dependent ways of learning that will serve them poorly outside institutional systems.

To make things far worse, those very structures that assist the traditional teacher (grades, compulsory attendance, fixed outcomes, concept of failure, etc) are deeply antagonistic to unteaching and are exactly why it is needed in the first place. Unteachers face a huge upstream struggle against an overwhelming tide that threatens to drown passionate learning every inch of the way. The results of unteaching can be hard to defend within a traditional educational system because, by conventional measures, it is often inefficient and time-consuming. But conventional measures only make sense when you are trying to make everyone do the same things, through the same means, with the same ends, measured by and in order to meet the same criteria. That’s precisely the problem.

The final nail in unteaching’s coffin is that it is applied very unevenly across the educational system, so every freedom it brings is counterbalanced by a mass of reiterated antagonistic lessons from other courses and programs. Every time we unteach someone, two others reteach them.  Ideally, we should design educational systems that are friendlier to and more supportive of learner autonomy, and that are (above all else) respectful of learners as human beings. In K-12 teaching there are plenty of models to draw from, including Summerhill, Steiner (AKA Waldorf) schools, Montessori schools, Experiential Learning Schools etc. Few are even close to perfect, but most are at least no worse than their conventional counterparts, and they start with an attitude of respect for the children rather than a desire to make them conform. That alone makes them worthwhile. There are even some regional systems, such as those found in Finland or (recently) British Columbia, that are heading broadly in the right direction. In universities and colleges there are plenty of working models, from Oxford tutorials to Cambridge supervisions, to traditional theses and projects, to independent study courses and programs, to competency-based programs, to PLAR/APEL portfolios, and much more. It is not a new idea at all. There is copious literature and many theoretical models that have stood the test of time, from andragogy to communities of practice, through to teachings from Freire, Illich, Dewey and even (a bit quirkily) Vygotsky. Furthermore, generically and innately, most distance and e-learning unteaches better than its p-learning counterparts because teachers cannot exert the same level of control and students must learn to learn independently. Sadly, much of it is spoiled by coercing students with grades, thereby providing the worst of both worlds: students are forced to behave as the teacher demands in their terminal behaviours but, without physical copresence, are less empowered by guidance and emotional/social support with the process. Much of my own research and teaching is concerned with inverting that dynamic – increasing empowerment and social support through online learning, while decreasing coercion. I’d like to believe that my institution, Athabasca University, is largely dedicated to the same goal, though we do mostly have a way to go before we get it right.

Why it matters

Unteaching is to a large extent concerned with helping learners – including adult learners – to get back to the point at which most children start their school careers – driven by curiosity, personal interest, social value, joy, delight – but that is schooled out of them over years of being taught dependency.  Once misconceptions about what education is for, what teachers do, and how we learn, have been removed, teaching can happen much more effectively: supporting, nurturing, inspiring, challenging, responding, etc, but not controlling, not making students do things they are not ready to do for reasons that mean little to them and have even less to do with what they are learning.

However, though it is an immensely valuable terminal outcome, improved learning is perhaps not the biggest reason for unteaching. The real issue is moral: it’s simply the right thing to do. The greatest value is that students are far more likely to have been treated with the respect, care, and honour that all human beings deserve along the way. Not ‘care’ of the sort you would give to a dog when you train it to be obedient and well behaved. Care of the sort that recognizes and valorizes autonomy and diversity, that respects individuals, that cherishes their creativity and passion, that sees learners as ends in themselves, not products or (perish the thought) customers. That’s a lesson worth teaching, a way of being that is worth modelling. If that demands more effort, if it is more fallible, and if it means that fewer students pass your tests, then I’m OK with that. That’s the price of admission to the unlearning zone.


Understanding the response to financial and non-financial incentives in education: Field experimental evidence using high-stakes assessments

What they did

This is a report by, Simon Burgess, Robert Metcalfe, and Sally Sadoff on a large scale study conducted in the UK on the effects of financial and non-financial incentives on GCSE scores (GCSEs are UK qualifications usually taken around age 16 and usually involving exams), involving over 10,000 students in 63 schools being given cash or ‘non-financial incentives’. ‘Non-financial incentives’ did not stretch as far as a pat on the back or encouragement given by caring teachers – this was about giving tickets for appealing events. The rewards were given not for getting good results but for particular behaviours the researchers felt should be useful proxies for effective study: specifically, attendance, conduct, homework, and classwork. None of the incentives were huge rewards to those already possessing plenty of creature comforts but, for poorer students, they might have seemed substantial. Effectiveness of the intervention was measured in terminal grades. The researchers were very thorough and were very careful to observe limitations and concerns. It is as close to an experimental design as you can get in a messy real-world educational intervention, with numbers that are sufficient and diverse enough to make justifiable empirical claims about the generalizability of the results.

What they found

Rewards had little effect on average marks overall, and it made little difference whether rewards were financial or not. However, in high risk groups (poor, immigrants, etc) there was a substantial improvement in GCSE results for those given rewards, compared with the control groups. 

My thoughts

The only thing that does surprise me a little is that so little effect was seen overall, but I hypothesize that the reward/punishment conditions are so extreme already among GCSE students that it made little difference to add any more to the mix.  The only ones that might be affected would be those for whom the extrinsic motivation is not already strong enough. There is also a possibility that the demotivating effects for some were balanced out by the compliance effects for others: averages are incredibly dangerous things, and this study is big on averages.

What makes me sad is that there appears to be no sense of surprise or moral outrage about this basic premise in this report.

dogs being whipped, from Jack London's 'Call of the Wild' It appears reasonable at first glance: who would not want kids to be more successful in their exams? When my own kids had to do this sort of thing I would have been very keen on something that would improve their chances of success, and would be especially keen on something that appears to help to reduce systemic inequalities. But this is not about helping students to learn or improving education: this is completely and utterly about enforcing compliance and improving exam results. The fact that there might be a perceived benefit to the victims is a red herring: it’s like saying that hitting dogs harder is good for the dogs because it makes them behave better than hitting them gently. The point is that we should not be hitting them at all. It’s not just morally wrong, it doesn’t even work very well, and only continues to work at all if you keep hitting them. It teaches students that the end matters more than the process, that learning is inherently undesirable and should only done when there is a promise of a reward or threat of punishment, and that they are not in charge of it. 

The inevitable result of increasing rewards (or punishments – they are functionally equivalent) is to further quench any love of learning that might be left at this point in their school careers, to reinforce harmful beliefs about how to learn, and to further put students off the subjects they might have loved under other circumstances for life.  In years to come people will look back on barbaric practices like this much as we now look back at the slave trade or pre-emancipation rights for women.

Studies like this make me feel a bit sick. 


Address of the bookmark: http://www.efm.bris.ac.uk/economics/working_papers/pdffiles/dp16678.pdf

‘Rote learning, not play, is essential for a child’s education’ – seriously?


An interesting observation…

Helen Abadzi, an expert in cognitive psychology and neuroscience, who was an education specialist at the World Bank, said that pupils who “overlearn” and repeatedly practise tasks, such as mental arithmetic, free up their working memory for more “higher order” analytical thinking.

Yes, they do, good point. We should not forget that. Unfortunately, she goes way beyond her field of expertise and explicitly picks on Sir Ken Robinson in the process…

“Go out and play, well sure – but is that going to teach me mental math so I can go to a store and instantly make a decision about what is the best offer to buy?” she said.

I cannot be certain but, as far as I know, and although he has made the occasional wild assertion, Sir Ken has never for one moment suggested that overlearning should be avoided. In fact, that’s rather obvious from the examples he gives in what the article acknowledges is the most popular TED talk of all time. I’ve yet to meet a good ballerina that has not practiced until it hurt. When you get into the flow of something and truly play, rote learning is exactly what you do. I have practiced my guitar until my fingers bled. Indeed, for each of my many interests in life, I have very notably repeatedly practiced again, again, and again, doing it until I get it right (or at least right enough). I’m doing it right now. I am fairly certain that you have done the same. To suggest that play does not involve an incredible amount of gruelling repetition and rote learning (particularly valuable when done from different angles, in different contexts, and with different purposes, a point Abadzi fails to highlight but I am sure understands) is bizarre. Even my cats do it. It is even more bizarre to leap from suggesting that overlearning is necessary to a wildly wrong and completely unsubstantiated statement like:

People may not like methods like direct instruction – “repeat after me” – but they help students to remember over the long term. A class of children sitting and listening is viewed as a negative thing, yet lecturing is highly effective for brief periods.

Where the hell did that come from? A scientist should be ashamed of such unsupported and unsupportable tripe. It does not follow from the premises. We need to practice, so extrinsic motivation is needed to make students learn? And play is not essential? Seriously? Such idiocy needs to be stamped on, stamped out, and stamped out hard. This is a good case study in why neuroscience is inadequate as a means to explain learning, and is completely inadequate as a means to explain education.

In the interests of fairness, I should note that brief lectures (and, actually, even long lectures) can indeed lead to effective learning, albeit not necessarily of what is being lectured about and only when they are actually interesting. The problem is not lectures per se, but the fact that people are forced to attend them, and that they are expected to learn what the lecturer intends to teach.

Activity trackers flop without cash motivation – Futurity


Another from the annals of unnecessary and possibly harmful research on motivation. Unsurprisingly, fitness trackers do nothing for motivation and, even less surprisingly, if you offer a reward then people do exercise more, but are significantly less active when the reward is taken away…

…at the end of twelve months, six months after the incentives were removed, this group showed poorer step outcomes than the tracker only group, suggesting that removing the incentives may have demotivated these individuals and caused them to do worse than had the incentives never been offered.

This effect has been demonstrate countless times. Giving rewards infallibly kills intrinsic motivation. When will we ever learn?

One interesting take-away is that (whether or not the subjects took more steps) there were no noticeable improvements in health outcomes across the entire experimental group. Perhaps this is because 6 months is not long enough to register the minor improvements involved, or maybe the instrument for measuring improved outcomes was too coarse. More likely, and as I have previously observed, subjects probably did things to increase their step count at the expense of other healthy activities like cycling etc. 

Recording of my TCC2016 keynote: The Distributed Teacher

This is the recording of my keynote at the TCC2016 online conference, on the nature of learning and teaching: the inherently social, distributed nature of it, why e-learning is fundamentally different from p-learning, and how we harmfully transfer pedagogies and processes from physical classrooms to online contexts in which they do not belong. If you want to watch it, skip the first 5 minutes because there was a problem with the sound and video (I hate you, Adobe Connect): the talk itself begins at a few seconds after the 5 minute mark.

Downloadable slides and details of the themes are at https://landing.athabascau.ca/file/view/1598774/the-distributed-teacher-slides-from-my-tcc-2016-keynote

Address of the bookmark: http://squirrel.adobeconnect.com/p1bvy7grca7/

Cheerful to a Fault: “Positive” Practices with Negative Implications – Alfie Kohn

One in a long series of excellent posts from Alfie Kohn, this time examining the problem of praise. The problem with praise and related things mostly only arises when you praise the person, not what they do. All too often it is a rather unpleasant means of asserting authority, and thus it causes a focus on meeting extrinsic goals, to the detriment of the intrinsic pleasure of doing something. We all need feedback, and it is great to know how we are doing through someone else’s eyes, but it’s much too easy for helpful reactions to turn into extremely unhelpful judgement, much too simple for that to reinforce or establish unhealthy power relationships, and absurdly easy for that to become the reason for doing something.

The post covers other issues too, notably the risks of too much focus on happiness and cheerfulness (neither of which are always appropriate responses to circumstances). I particularly like his translation of “Only Positive Attitudes Allowed Beyond This Point.”  as meaning “My Mental Health Is So Precarious That I Need All of You to Pretend You’re Happy.”

Address of the bookmark: http://www.alfiekohn.org/blogs/cheerful

Over two dozen people with ties to India’s $1-billion exam scam have died mysteriously in recent months

“… the scale of the scam in the central state of Madhya Pradesh is mind-boggling. Police say that since 2007, tens of thousands of students and job aspirants have paid hefty bribes to middlemen, bureaucrats and politicians to rig test results for medical schools and government jobs.

So far, 1,930 people have been arrested and more than 500 are on the run. Hundreds of medical students are in prison — along with several bureaucrats and the state’s education minister. Even the governor has been implicated.

A billion-dollar fraud scheme, perhaps dozens murdered, nearly 2000 in jail and hundreds more on the run. How can we defend a system that does this to people? Though opportunities for corruption may be higher in India, it is not peculiar to the culture. It is worth remembering that more than two-thirds of high school Canadian students cheat (I have seen some estimates that are notably higher – this was just the first in the search results and illustrates the point well enough):

According to a survey of Canadian university & college students:

  • Cheated on written work in high school 73%
  • Cheated on tests in high school 58%
  • Cheated on a test as undergrads 18%
  • Helped someone else cheat on a test 8%

According to a survey of 43,000 U.S. high school students:

  • Used the internet to plagiarize 33%
  • Cheated on a test last year 59%
  • Did it more than twice 34%
  • Think you need to cheat to get ahead 39%

Source: http://www.cbc.ca/manitoba/features/universities/

When it is a majority phenomenon, this is the moral norm, not an aberration. The problem is a system that makes this a plausible and, for many, a preferable solution, despite knowing it is wrong. This means the system is flawed, far more than the people in it. The problems emerge primarily because, in the cause of teaching, we make people do things they do not want to do, and threaten them/reward them to enforce compliance. It’s not a problem with human nature, it’s a rational reaction to extrinsic motivation, especially when the threat is as great as we make it. Even my dog cheats under those conditions if she can get away with it.  When the point of learning is the reward, then there is no point to learning apart from the reward and, when it’s to avoid punishment, it’s even worse. The quality of learning is always orders of magnitude lower than when we learn something because we want to learn it, or as a side-effect of doing something that interests us, but the direct consequence of extrinsic motivation is to sap away intrinsic motivation, so even those with an interest mostly have at least some of it kicked or cajolled out of them. That’s a failure on a majestic scale. If tests given in schools and universities had some discriminatory value it might still be justifiable but perhaps the dumbest thing of all about the whole crazy mess is that a GPA has no predictive value at all when it comes to assessing competence.

Address of the bookmark: http://www.theprovince.com/health/Over+dozen+people+with+ties+India+billion+exam+scam+have+died/11191722/story.html