The importance of a good opening line

Posted

This post asks the question,

How does the order of questions in a test affects how well students do?

The answer is “significantly.”

The post points to a paywalled study that shows, fairly conclusively, that starting with simpler questions in a typical academic quiz (on average) improves the overall results and, in particular, the chances of getting to the end of a quiz at all.  The study includes both an experimental field study using a low-stakes quiz, and a large-scale correlational study using a PISA dataset. Some of the effect sizes are quite large: about a 50% increase in non-completions for the hard-to-easy condition compared with the easy-to-hard condition, and a about a 25% increase in time on task for the easy-to-hard condition, suggesting students stick at it more when they have gained confidence earlier on. The increase in marks for the easy-to-hard condition compared with the hard-to easy condition is more modest when non-completions are excluded, but enough to make the difference between a pass and a fail for many students.

I kind-of knew this already but would not have expected it to make such a big difference.  It is a good reminder that, of course, objective tests are not objective. A quiz is a kind of interactive story with a very definite beginning, middle, and end, and it makes a big difference which parts of the story happen when, especially the beginning. Quizzes are like all kinds of learning experience: scaffolding helps, confidence matters, and motivation is central.  You can definitely put someone off reading a story if it has a bad first paragraph. Attitude makes all the difference in the world, which is one very good reason that such tests, and written exams in general, are so unfair and weak at discriminating capability, and why I have always done unreasonably well in such things: I generally relish the challenge. The authors reckon that adaptive quizzes might be one answer, and would especially benefit weaker students by ramping up the difficulty slowly, but warn that they may make things worse for more competent students who would experience the more difficult questions sooner. That resonates with my experience, too.

I don’t give marks for quizzes in any of my own courses and I allow students to try them as often as they wish but, even so, I have probably caused motivational harm by randomizing formative questions. I’m going to stop doing that in future. Designated teachers are never the sole authors of any educational story but, whenever they exert control, their contributions can certainly matter, at small scales and large. I wonder, how many people have had their whole lives changed for the worse by a bad opening line?

Source: It’s a question of order – 3-Star learning experiences

 

Over two dozen people with ties to India’s $1-billion exam scam have died mysteriously in recent months

Posted , updated on

“… the scale of the scam in the central state of Madhya Pradesh is mind-boggling. Police say that since 2007, tens of thousands of students and job aspirants have paid hefty bribes to middlemen, bureaucrats and politicians to rig test results for medical schools and government jobs.

So far, 1,930 people have been arrested and more than 500 are on the run. Hundreds of medical students are in prison — along with several bureaucrats and the state’s education minister. Even the governor has been implicated.“

A billion-dollar fraud scheme, perhaps dozens murdered, nearly 2000 in jail and hundreds more on the run. How can we defend a system that does this to people? Though opportunities for corruption may be higher in India, it is not peculiar to the culture. It is worth remembering that more than two-thirds of high school Canadian students cheat (I have seen some estimates that are notably higher – this was just the first in the search results and illustrates the point well enough):

According to a survey of Canadian university & college students:

  • Cheated on written work in high school 73%
  • Cheated on tests in high school 58%
  • Cheated on a test as undergrads 18%
  • Helped someone else cheat on a test 8%

According to a survey of 43,000 U.S. high school students:

  • Used the internet to plagiarize 33%
  • Cheated on a test last year 59%
  • Did it more than twice 34%
  • Think you need to cheat to get ahead 39%

Source: http://www.cbc.ca/manitoba/features/universities/

When it is a majority phenomenon, this is the moral norm, not an aberration. The problem is a system that makes this a plausible and, for many, a preferable solution, despite knowing it is wrong. This means the system is flawed, far more than the people in it. The problems emerge primarily because, in the cause of teaching, we make people do things they do not want to do, and threaten them/reward them to enforce compliance. It’s not a problem with human nature, it’s a rational reaction to extrinsic motivation, especially when the threat is as great as we make it. Even my dog cheats under those conditions if she can get away with it.  When the point of learning is the reward, then there is no point to learning apart from the reward and, when it’s to avoid punishment, it’s even worse. The quality of learning is always orders of magnitude lower than when we learn something because we want to learn it, or as a side-effect of doing something that interests us, but the direct consequence of extrinsic motivation is to sap away intrinsic motivation, so even those with an interest mostly have at least some of it kicked or cajolled out of them. That’s a failure on a majestic scale. If tests given in schools and universities had some discriminatory value it might still be justifiable but perhaps the dumbest thing of all about the whole crazy mess is that a GPA has no predictive value at all when it comes to assessing competence.

Address of the bookmark: http://www.theprovince.com/health/Over+dozen+people+with+ties+India+billion+exam+scam+have+died/11191722/story.html

Exam focus damaging pupils' mental health, says NUT – BBC News

Posted , updated on

A report on a survey of 8,000 teachers and a review of the research.

The report sponsors observe…

“Many of the young people Young Minds works with say that they feel completely defined by their grades and that this is very detrimental to their wellbeing and self-esteem.”

It seems that at least some of their teachers do indeed (reluctantly) define them that way…

One junior school teacher said: “I am in danger of seeing them more in terms of what colour they are in my pupils’ list eg are they red (below expectation), green (above expectation) or purples (Pupil Premium) – rather than as individuals.”

Indeed, it appears to be endemic…

Kevin Courtney, deputy general-secretary of the NUT, said: “Teachers at the sharp end are saying this loud and clear, ‘If it isn’t relevant to a test then it is not seen as a priority.’

“The whole culture of a school has become geared towards meeting government targets and Ofsted expectations. As this report shows, schools are on the verge of becoming ‘exam factories’.”

He argued the accountability agenda was “damaging children’s experience of education”, which should be joyful and leave them with “a thirst for knowledge for the rest of their lives”.

This is terrible and tragic. So surely the British government is trying to do something about it? Not so much…

A Department for Education spokesperson said: “Part of our commitment to social justice is the determination to ensure every child is given an education that allows them realise their potential.

“That’s why we are raising standards with a rigorous new curriculum, world class exams and new accountability system that rewards those schools which help every child to achieve their best.”

Helping people to realise their potential is a noble aim. A “rigorous new curriculum, world class exams and new accountability system” is a guaranteed way to prevent that from happening. Duh. Didn’t those that run the UK government learn anything in their expensive private schools? Oh…

Address of the bookmark: http://www.bbc.co.uk/news/education-33380155

The death of the exam: Canada is at the leading edge of killing the dreaded annual ‘final’ for good | National Post

Posted , updated on

Good news!

There’s not much to disagree with in this article, that reports on some successful efforts to erode the monstrously ugly blight of exams in Canada and beyond, and some of the more obvious reasoning behind the initiatives to kill them. They don’t work, they’re unfair, they’re antagonistic to learning, they cause pain, etc. All true.

Address of the bookmark: http://news.nationalpost.com/news/canada/the-death-of-the-exam-canada-is-at-the-leading-edge-of-killing-the-final-for-good