Alternative Grading Scales

I've been asked to be part of the Observation of Learning and Teaching team at my workplace, alongside acting as mentor for some new teachers. I love watching other teachers strut (or fail to strut) their stuff in class - it's always fascinating to see different group dynamics, and an opportunity to pick up ideas for teaching that I may nebver have considered, or indeed be reminded of things that I've forgotten or not used in a while. Last year's observations for my DELTA were great, although I (probably fortunately) didn't see any car crash jobs. A shame in a way, as a lesson Where All Your Shit Goes Tits Up makes for excruciatingly fun viewing, as well as giving a salutary example inasmuch as a) things can go belly up for anyone and b) things going wrong are often a better teacher of technique than things going right. My favourite buggered-up lesson involved a teacher losing the plot, getting sidetracked, then eventually pretending to be a seagull before perching on a table, squawking.

Anyway, being an observer inevitably involves paperwork, and feedback. Oh, lovely feedback: that moment when someone smiles at you as you both sit down in a room lit by fluorescent strip lighting, smiles even more brightly and says, 'So, how did YOU think your lesson went?', the you say what you thought, and hope that you aren't going to get a Shit Sandwich* back.

A lot of these feedback sessions involve someone saying, 'on a scale of one to ten, how would you rate that?', or something similar. B.O.R.I.N.G.
I was thinking about this the other day, and started wondering whether you could scale things in a different way. After all, getting teachers to be imaginative can only be a good thing, right?
So why not ask them something like this?

'On a scale of monday morning to saturday night, how good was that lesson?'

When you start to think about it, this makes a lot of sense, in a leftfield kind of way. An observed lesson should, in theory at least, be a typical lesson - how the teacher delivers the class on average, rather than the manically prepared, coiffed, perfumed, shaven, dolled-up confection of  a lesson that is the norm when you know that someone's coming in to take notes. I'd say, on this scale, anywhere between thursday at 11.30 a.m. and friday at 7 pm (with another bump on saturday between 1 and 6.30 pm) means it's a good lesson. Anything before thursday would be dull or below par; anything after friday 9.30 pm would be loud and overblown;  and saturday at 3 a.m. would be a lesson that's hoiking kebabs into the gutter.
Anyway, I'm going to give this a try, and devise some for marking student essays.
Probably something along the lines of

'On a scale of Beaker from the Muppets to Animal, how incomprehensible is this essay?'.

*Shit Sandwich: the act of an appraiser giving you feedback where they start and finish with positive comments, filling the bit in between with a steaming pile of criticism, invective and bile, as in:
'Well, I liked your class file. Unfortunately, your lesson went down worse than a drunken whore, and you are, quite frankly, a stain upon the entire teaching profession. However, I was glad to see that you have a nice tie on.'

Mind the Gap.

Here's a question for you - what do the following all have in common?

• A cryptic crossword puzzle
• A cloze (aka gap-fill) exercise
• A sudoku puzzle
• A grammar exercise where you have to write the correct form of the verb
• An algebra exercise
• one of those team-building things where you have to work out how to cross a river using a piece of string and two dead dogs or something
• a reading task asking you to identify words and phrases in context that mean the same as a given set of synonyms

That's right, they're all problem solving tasks.
So why am I bringing it up?
The point is this: do the ones about language actually test a knowledge of language, or do they in fact only test an ability to solve a problem? In other words, it strikes me that many of the tasks in student workbooks are not real tests of language knowledge whatsoever, but exercises in learning skills.
Let me give an example task to you.
Here are three rules.
If a sequence of numbers is 3 digits and ends in 9, follow it with 12.
If a sequence of numbers is 3 digits and ends in 7, change the 7 to 9 and add 12.
If  a sequence of numbers is 4 or more digits, it must be preceded by 21.
and here are some sequences:

1. 329
2. 5437
3. 777
4. 919
5. A427
6. 3424245539

Easy, isn't it?
Now change the rules to those describing how to make comparative adjectives in English.
The simple fact is that many of the exercises we do with our students do not in fact test their understanding of language, but their cognitive and problem-solving capacities. Someone who can understand a logic problem, as long as the problem and a model solution is clearly presented, should be able to solve any given issue. Now, while it may be useful for someone to comprehend a given set of rules, it does not necessarily follow that that person is in fact capable of using the language in a way that is comprehensible, simply because languages have a nasty habit of not following their own laws. This is why, whenever we do level testing, we should always look at a suite of abilities rather than rely on the good old grammar test prior to deciding a language level. It also explains, by the way, why EFL students tend to score higher on formalised language tests which are generally problem-solving based tasks, than ESOL students.

SATs, contractors and doing it on the cheap

Off the topic of ELT, but still educationally related - The SATs(Standard Attainment Tests), sat by 11 and 14-year-olds in the UK. There's been a bit of a kerfuffle, to put it mildly, at the delay in releasing the results this year. There's been more disgruntlement at schools, where teachers have been resending marked papers because of inaccuracies in the scoring. So, what's going on?

the papers have been sent off to be marked to a company contracted to do it, presumably the cheapest one available.

Its name?
ETS Europe.

The marking of the SATs has been subcontracted to a company not based in the UK. In other words, British students' work is being checked and marked for accuracy in somewhere other than the UK.

I can't begin to describe how shocking I find this. It's wrong on so many levels. Now, if it was just a case of a multiple choice paper being fed into a computer, I could accept that. If it was just checking the result of a maths paper, where you can only have a correct or wrong answer, I could just about accept that too. But it seems (and honestly, I couldn't be more glad if I were wrong on this point) that the entire lot is being sent off. OK, you can make a point abour objectivity in marking: The examiner will have a set number of descriptors against which he/she will check the submitted work, and based on that assign a mark. However, I can see so many ways that marking will be inaccurate.
Just a few examples:
Orthography. The way that UK kids are taught to write is significantly different from the way it is done in other EU countries. This is a fairly neat example:
, and it's written by an adult! Imagine an 11-year-old's being deciphered by an examiner.

Cultural and Social mileau. Taken out of context, how can anything relating to a culture or society be accurately interpreted, let alone assigned a score in an exam?

Examiner's L1/L2 competency. No matter how good the examiner's English may be, nevertheless they will be marking and interpreting at one remove - that is, they will have to decode the information, recode into L1, interpret according to two sets of cultural and possibly sociolinguistic filters, then assign a mark and re-encode into English. There is no way it can be done entirely fairly, as any examiner in this situation will use affective filters in the process.

All I can say is that it's WRONG. Totally bloody WRONG.