Fuck no. Open book tests are the cow pat in the field of quantitative knowledge assessment.
The Oxford, Cambridge and RSA (OCR) exam board wants pupils to be allowed to use Google to search for answers in exams. The proposal has been put forward by Mark Dawe, the OCR exam board chief executive, who claims that using Google is akin to using a calculator in mathematics and that there should be Google and non-Google …
To be checking the average comprehension skills, spelling and grammar usage in these very comments and drawing conclusions as to the utility of non-quantitative knowledge assessment techniques in the education of the commenters over the last thirty years, iniquitous soft one!
The clue is in the name, Open Book. Not phone-a-friend (Facebook), not ask-the-audience (Twitter). For exam purposes you'd need a whitelist of acceptable resources. It's an awful lot of effort on the part of the examiners, with the aim of solving a problem which hasn't been proven to exist.
"with the aim of solving a problem which hasn't been proven to exist"
Actually, the problem has been demonstrated to exist, which is that a good proportion of exams favour those better at rote learning of facts than solving problems, and for many subjectively assessed subjects also favour those who can write quickly. Einstein comes to mind as the finest example of somebody who was not served well by such approaches to assessing ability, possibly because he was dyslexic.
I can see some value in both rote learning, fact retention, and the ability to write quickly and coherently, on the other hand it means that in most subjective assessments these capabilities will trump people of superior intellect and problem solving who can't write quickly or struggle to remember things you'd look up in the real world anyway.
> Einstein comes to mind as the finest example of somebody who was not served well by such approaches to assessing ability, possibly because he was dyslexic.
Please stop with this urban myth. Einstein did very very well at school and was in fact top of his class in most subjects.
We've rewarded those who can learn by rote and parrot facts, and then probably forget them as soon as they leave the room.
Those who can integrate information, draw conclusions and understand implications of data are the ones we need. But the exam system penalises them. And it starts very early on (<11).
The biggest joke ( for the younger kids) is that the current govt. want them to rote learn their 12x table for SATS.
I can remember 12d to the shilling.
And 12 inches to a foot.
But I don't need to calculate them anymore.
And labelling a kid a failure if they can't remember that 7x12=84, when being able to work out 7x12= 7(2+10) is much more useful amounts to sheer lunacy. But it's a touchstone for how we use the exam system.
Would a better approach not be to have a system where all of the stuff they access on-line is included in the submission the examiner gets?
That way you can mark how well they "used" google and check for simply asking the question and/or using google to go to a 'mechanical Turk' site for a solution.
OK, would make marking a bit more tedious, but maybe the knowledge that their search operations are assessed would make for a more focused approach.
> we were allowed to bring in whatever we wanted and it would not have helped in the slightest
Yeah, I've had that one too. So I brought a picture of the head lecturer's wife making an unexpected (for her) visit to my pig farm.
Turns out the old geezer wasn't as attached to her as we thought. :-(
It's a good idea, tbh.
Googling any question is going to give you dozens of correct and incorrect answers, and the ability to assess them and isolate which ones are right is arguably closer to what most jobs involve day-to-day than the capacity to memorize large amounts of irrelevant information.
I always found open book exams harder than closed book, anyway - the questions are written to be harder because the kids have access to the information. Closed-book exams didn't even really require revision if you could remember a few basic principles to work the answer out with. I imagine that you could do the same with Google - let kids Google the answers, but pose interpretative or qualitative questions. Being able to search for, find and compare multiple solutions and pick the best one is more use than simplistic black-and-white questions with right answers, wrong answers and bonus points for including your working out.
if you google something and it 50 / 50 on correct / incorrect answers and you haven't learned the correct answer (why would you, just google it) - how do you know which one to choose?
In several areas of interest to me the top 10 google results are all incorrect - mainly because people remember wrong and post things online - what do you do then?
Yeah agree here, even if you are working out formulas a rough idea of what you should be getting eg an energy output, or amount of force, comes from what you know and its been enough to stop me making a mistake when i have accidently added one zero to many on a calculation.
they are quite challenging. But by design (joke!).
Recognising and being able to prove mathematics is incorrect (i.e. they give an incorrect formula for you to solve the problem with), is really a test in the understanding of proof by induction.
One would think that software could be tested that way.
Write a software algorithm that is memory efficient at sorting numbers.
How would you make it faster if memory was not an issue?
If you know the distribution but not the specific elements of data (say ASCII text), could you design a faster algorithm?
If not ,explain why not and the cases where solving this might be useful in society.
He argues that in everyday life, pupils will be free to use Google and that this “open book” approach should be extended to examinations, arguing that examinations are about understanding and interpretation
Then why have exams*? In everyday life since starting full-time work I've never needed to write six essay-style answers in three hours flat.
* I may be biased, since I never could cram for exams and always did worse in them than in any course work.
Now, I'm precisely the opposite. I'm really, really, good at passing exams. This includes programming in C++ - something that in real life I'm complete rubbish at, being a certified system architect in xxxx's software - something else I'm crap at, etc.
In my opinion, exams are unfair, they suit people with my sort of short-term retention and penalise far better people who just aren't good in exam situations.
Well said. Sandman. But sadly it's also the same people who have done well out of short term rote learning who slide into positions of authority. And they are not going to give up on, or argue against ( unlike your good self) what made them successful.
The pointy haired bosses want to stay as bosses.
And no politician ever got elected by arguing for a creative answer.
Same here, I aced exams and all through primary received top marks because end of term exams comprised 100% of our grades. When I rolled into secondary school and homework (which I never did) and class projects started to be factored into out grades, my marks dropped like a stone.
The main reason for this restriction lies in its comedy/schadenfreude value, not to make The Apprentice a true test of the candidates' Business Acumen.
At least, I assume that that's the reason they all come across as such total fuckwits. It couldn't possibly be that they were all selected from the bottom of the pile, shirley?
This post has been deleted by its author
Many a truth spoken in jest.
There are a few of things here:
1. I do need to know how to acquire facts which suitable for short-term retention, during which time I can make particular decisions. I don't need to know the company's financial results forever, but I may need them while I make a plan for the next few years.
2. Learning to deal with stress. For example, go outside and do some exercise to get some balance, instead of focussing purely on the most immediate problem.
3. If you don't at least TRY to learn some facts, you won't learn any. It would have saved quite a few lives if Tony Blair remembered a couple of quotes from his history - "Don't get into a land war in Asia", "Iraq is not governable by civilised means" (thus said British and US military personnel in the past).
4. Learning to deal with failure. You really want them to learn this before the final exams, but dealing gracefully with failure is a key skill.
The upshot is, analysis without facts is useless. Facts without analysis is also useless. Academically, we need both. There are also non-academic faculties which children need to develop.
I give the internet generally a thumbs-down for schools. I know how I use it and I know how my kids use it. It doesn't teach much except cut & paste. If you can't use it to solve an exam question, why make it available? If you're opposed to rote learning, why is the application of that learning worthwhile to test?
Why on earth would we want to add the complexity, fragility and cost of IT to the exam process? Pen & paper work really well for the required application.
I think there should be a "effective use of data retrieval techniques" exam, where using search engines is approved, but for others no. There is no doubt using these tools are an important part of life, and that will only grow. However much like when I were a lad, we had a mathematics exam where we could use a calculator, and an arithmetic exam where we couldn't - I'd leave google to one (or possibly 2 subjects), but not to all.
Less seriously, perhaps they could introduce use of google as sort of a joker card, to be used on no more that 2 exams? ;)
Or an educated entrepreneur could research the last five years-worth of questions, prepare model answers, and have them for sale during the exam. Students could pre-register (with parents' credit card) and be able to log in for the goodies, and the real test to see if Junior was smart enough to memorise daddy's credit card number of PayPal login so as to allow him to pay at point of exam.
I love the "why memorize, just look it up" mindset that has taken hold of the public education system.
It's a root cause of why no-one under the age of fifty five can make change on demand, and why the bank teller I got into it with a few years back couldn't look at a column of amounts and see from the amounts involved that his calculator-derived result could not possibly be right.
I intervened only a few weeks back when a group of young high school seniors were trying to figure the 15% tip on a four person bill, and were at the stage of realizing that in the whole group, no-one knew how to work the tip calculator app on their late-model iPhones (each senior had one of course). In a place where we have an 8 and a bit percent sales tax there was an easy tip I could give them to prevent the waitress being stiffed, but the fact that not one of them could spitball a 15% tip from a bill submitted in a decimal coinage system quite frankly boggled my mind.
Yes, you can always "look it up" (except when you can't, but we'll not talk about that case because gosh, you have internet everywhere except when you don't and it always has the truth on it except when it doesn't) but there's no comprehension behind that mindset, only the surefire prediction that it won't be too long before we need to cater for people looking up how to look stuff up.
Wait ... isn't that the service Google aspires to be?
Obligatory Wikipedia reference for people to look up:
(Provided to ironically illustrate the difference between the actual experience and "looking it up". This note added when I realized no-one under fifty-five would likely get the reference and would assume the Wikipedia entry *was* the experience).
No, it's just the opposite. Short term rote learning is very much still what the exam system is about.
But those under and over 50 who did well and got good jobs are quite often the ones who promptly forgot everything they learnt as soon as they put down their pens.
What is important, Google or no Google, is that the students should be able to apply knowledge.
So the exam questions would need to be written to test understanding.
In a sense the factual information could even be provided on the test paper, if the question was taxing of ability to use it.
> But those under and over 50 who did well and got good jobs are quite often the ones who promptly forgot everything they learnt as soon as they put down their pens.
The purpose of an examination is not only to quantify your innate intelligence and understanding, but how well you can apply yourself in your chosen field. The person who can apply themselves to rote learning is likely to be the person who will apply themselves in their career.
> In a sense the factual information could even be provided on the test paper, if the question was taxing of ability to use it.
It has been my experience that when new graduates enter the workplace, they are often content to submit work that would garner a 'passing grade' in an academic setting rather than making an all-out effort. To prepare students for a work environment then, if the requisite factual information was included in a test paper, the pass mark threshold should be set at a much higher level than in conventional exams.
>"The purpose of an examination is not only to quantify your innate intelligence and understanding, but how well you can apply yourself in your chosen field. The person who can apply themselves to rote learning is likely to be the person who will apply themselves in their career."
So you equate being able to memorise, regurgitate and forget with "applying themselves". And consider that this is the key skill.
That doesn't hold water.
To start with, slogging away for a couple of years is not the same as growing and developing for a lifetime.
Then in that group of rote learners are those who are just naturally good at recall irrespective of other abilities such as understanding or indeed preseverance as well as those that do indeed have perseverance and can apply themselves, slogging away until they master the subject, but it does not include those who have aptitude, insight, creativity or even many who have genuine interest or commitment, but who just aren't good at churning out preformed answers to artificial questions.
Further traditional exams are a peculiar form of pressure that some deal with bettter than others. And while there are definitely some roles that routinely require you to find answers under pressure most do not and will give you time to think, reflect and investigate solutions.
> So you equate being able to memorise, regurgitate and forget with "applying themselves". And consider that this is the key skill.
> those who have aptitude, insight, creativity or even many who have genuine interest or commitment, but who just aren't good at churning out preformed answers to artificial questions.
In order to be proficient in any language it is necessary to build a vocabulary. High school (and even undergraduate level) study is largely concerned with building a student's vocabulary in a given subject - and this is equally true of languages, humanities and sciences.
Unfortunately there is no shortcut to acquiring the requisite 'vocabulary'. This process almost always requires memorisation, whether it be amo, amas, amat, the date of the Norman conquest or the definition of a halogen. Even the highest placed Harley Street medic was required to undergo tons of rote learning as an undergraduate.
Certainly, understanding is crucial (the meaning of words in the vocabulary), but it is only after internalising the shared vocabulary that people with aptitude can progress to being creative with the 'language' of a given course of study.
"What is important, Google or no Google, is that the students should be able to apply knowledge."
Knowledge implies you know things which in turn means you, at some point, memorized what you know. Otherwise, you don't know it.
At what point does the "open book" thinking examiner draw a line and say "you should be able to call upon this knowledge at will in order to do what we are teaching you to do"?
A physical chemist must have a passing familiarity with the gas laws, various mathematical techniques and so forth. He or she is no bloody use to anyone if those have to be looked up when needed.
And the despised rote learning lets me do arithmetic tricks in a rum, pizza and fag-addled 60 years maltreated brain that my 22 year old believes are damn near miraculous.
Dear me no, let us expunge rote learning of multiplication tables in first grade because they are of no use at all and anyway, you can always use your phone to do the math.
And if anyone thinks that internet access during exams will result in anything other than a tsunami of cut'n'paste plagiarism, well, I have this bridge for sale.
I agree. Some stuff has to be learnt. Particularly concepts and methods.
But that's not the same as saying all/most of what we assess has to be rote learned facts.
Too many of the arguments in this thread are based on reductio ad absurdium and "straw man" arguments.
Attacking the concept that we should use open book/internet sources as if this was proposed as the only exam type.
But maybe that's because those people are the ones that got where they are through rote learning and so aren't very sharp in the fields of reasoning and judgement.
> Short term rote learning is very much still what the exam system is about.
Perhaps because those in control of the exams are of the same mindset as though who dislike rote learning?
I don't have a great memory for my o-level material, but A-levels? Two years studying three subjects? Yes, I remember a great deal of that 30 years on. Of course, if you broaden the field of knowledge, you're likely to make it shallower and if you split learning up into short-term "units" with exams at the end of each one, or if you have coursework where you only need to remember things as long as it takes to put it into MS Publisher, then yes, you're unlikely to learn much for the long term. Even at o-levels, there was some benefit to learning the discipline of learning, even if the subject material was broad and not likely to be pursued later.
Personal bias here, but I suspect the political desire to get all children into University (as if everyone was the same) and the desire of Universities to attract more customers (now they are all fee-paying) conspired to create an academic abomination. Worse though is the competition for funding which causes academics to say outrageous things as they compete to get noticed on the academic x-factor grant-fund challenge.
One of the exams for one of my computer science modules at uni was done on computer with full access to the internet (it was a practical examination where we had to write code and xlst transforms etc).
I wasn't a big fan mostly because it's very easy in that circumstance to do timing on the questions. One of my friends also managed to give the wrong arguments to tar at the end of the exam and managed to overwrite all his work....
I don't think this is a good idea at all and I can't see anyway this can be done on a larger scale (ie national examinations) while still locking down on cheating and help from the outside.
"One of my friends also managed to give the wrong arguments to tar at the end of the exam and managed to overwrite all his work...."
Damn! If only your friend had had to rote-learn the most useful forms of the tar invocation.
Curse you, inevitable consequences!
Actually, I can never remember that bloody thing myself. Don't use it much. Have to look it up in mi' trusty Unix in a Nutshell when I do.
Of course this already exists - it is called course work, with the added bonus the time constraints don't have to be as strict (depending on how organised the pupil is or if they leave it to the last minute. Education and exams have a natural pendulum action between "course work makes it too easy restrict it and do exams" and "exams unfairly favour those good at fact retention and writing fast course work is far fairer". Personally I'd say course work reflects the real world of work far more accurately but then I also think the point of university is for academic people to be academical and advance the boundaries of knowledge so it all comes down to what you think the purpose of school, exams and university is for.
In principle I don't object to the kiddywinks being able to use a search engine, the issue is will they understand WHAT they are reading. Its all well and good copying something verbatim but not being able to understand it is not a good thing.
Besides, I bet you any money this scenario is common:
"Dad, whats polyunsaturated mean?"
dad, "dunno son, have you googled it?"
So little johhny knows WHAT it is but not WHY it is.
That's my gripe...
When I was at school, we were not allowed to use calcs in maths exams, about a year later, the rules were relaxed so they kiddies could use calcs....
An actual maths examination doesn't get any easier if you have a calculator.
For a start, it rarely involves many numbers. If you're at an intermediate stage all it will allow you to do is skip the chore-work of, for example multiplying all the elements of two matrices by hand. Not that you shouldn't be able to do that if needed, but it's at a level so far below that which you're actually testing that to enforce it only serves to make maths feel even less appealing to those who are not inclined to love it.
If you're testing arithmetic on the other hand, calculators are right out.
In some subjects, at some levels: maybe. Otherwise: just another way to make the population stupider. Some things - the basics, just have to be learnt or understood and not simply searched for. Just like primary school children are not allowed (i hope) calculators but then at a higher level calculators become a necessity. Moreover, in some subjects this would completely destroy the whole point of some questions (e.g unseen pieces in literature, where you're supposed to make up your own assessment instead of regurgitating something you prepared beforehand)
Most comments seem to be against this, but in the real world this is how they'll be looking up information. Testing in this way will force them to be better about how they Google, what sites they consider valid and why, etc.
Not all tests have to be this way, but some should, just like you want people to understand some math without a calculator but also know how to use a calculator since no one is going to do long division by hand when they have a calculator available.
Then give them access to a research intranet with a pet search engine. Keep them the hell away from the Internet. You could *write* a student that could pass this exam, including imaging the paper and manipulating a pen to write the answers if necessary.
Question 1: Post this onto a waiting Internet forum and offer money via paypal for answers from Uni students who passed this exam.
Answer: Pass with flying colors without understanding a word you wrote, relegating non-cheaters to the bottom of the table they would otherwise have been relatively high up, teaching them a valuable life lesson and screwing your country's future.
Unusually I've posted several times in this thread.
Because I'm a teacher, and a parent of teenagers.
But, also because I believe quite passionately that the "dumbing down" is not prevented by mechanical rote learning; it's caused by it.
Creating a generation who slog along, marching in step, learning their lessons by heart and regurgitating them without thought.
Yes we need to learn and know stuff.
But that's not the same as building our education and exam system on great mounds of memorised facts that can be forgotten as soon as the exam is passed
I do wonder if we could do away with exams completely. Instead you would have to do 2 sets of coursework maybe even for 2 different teachers to get an averaged score. Assuming this was possible it would mean that kids are set problems which could be varied (created by the teacher/school) and within acceptable limits for research the kids would have to produce their work.
The main reason I think this could work is the lack of CV's or skills in writing a CV. Recently I had to help a young brother in law write a CV because he had nothing to put on it. If schools became more problem oriented in their assessments it would instantly give the kids something to write about as achievements.
And of course I am not a fan of exams anyway. I am dyslexic and it gives me a disadvantage even on topics I know very well. On the other hand kids with no clue but a good memory could pass exams while knowing barely anything about the topic.
I am sure there are problems with such an idea but just a thought.
Different exams test different things. An open book exam is good at testing ability to problem solve and come up with creative answers. A "normal" exam is good at testing fact retention and relevance.
Both are good in appropriate settings.
Obvious statements are obvious, but don't appear to have been made yet. And can we please, on a technical site, avoid the confusion the government/media have between arithmetic and maths.
I always thought exams are a bad metric when grading a person, the best idea is to use coursework and constant monitoring, sure you need exams, just ensure the exam paper itself has all the information required....asking someone to remember an equation is not that important, but being able to use an equation when given is...
Biting the hand that feeds IT © 1998–2021