ChatGPT talks its way through Wharton MBA, medical exams OpenAI's chat software ChatGPT, if let loose on the world, would score between a B and a B- on Wharton business school's Operations Management exam, and would approach or exceed the score needed to pass the US Medical Licensing Exam (USMLE). While this may say more about the static, document-centric nature of testing material …
It's already happening at Microsoft, Google, Facebook...
But they will soon realise that the AI is the ultimate "wally" who does not care about its job (for it has no self-awareness and therefore no self-interest) and simply continues to generate the same rubbish quality of bullshit when you tell it that it's a crap employee.. And they can't sack it anyway.
does not care about its job (for it has no self-awareness and therefore no self-interest) and simply continues to generate the same rubbish quality of bullshit when you tell it that it's a crap employee
We had one of those once....I think his name was Steve.
Sounds like it's about time to fire all those useless CEOs, like Tim Cook or Elon Musk, with their multi ten and hundred million dollar per year compensation packages, as AI has arrived at a point where it's perfectly capable of replacing them, for the cost of a PC and the electricity it consumes.
Then again, these people seem not to be there to make actual business decisions anyway, but to act as circus clowns for their respective companies and to generate attention to those. And there's no news of ChatGPT having mastered also the circus clown exam, yet.... so....
"I'd say it's perfect for... politician"
Given that there are surely vast amounts of political theory literature in ChatGPT's training set, I wonder if, given a certain political direction, ChatGPT could write a coherent series of policy documents better than an actual politician (of which there are very few these days who decide their policies based on sound principles rather than what is expedient at the time or what is whispered in their ears by lobbyists as brown envelopes change hands)
Are they still coming or already here in Systems Physical Command with Remote Controls/Programming Software Driven Hardware Utilities?
..decided to put ChatGPT, an arguably amoral automated advisor and factually-challenged expert system, to the test.
An unearthly alien encounter probing for inherent catastrophic weaknesses and attacking and countering human defences with the likes of a ChatGPT or Copilot AIMaster Pilot would be testing and factually challenging of expert and arguably amoral autonomous anonymised advisors ...... and can you imagine the true gems of animal intelligence and pearls of ancient barbaric wisdom uncovered and displayed in denial of the truth of those increasingly revelatory virtual forays/sorties/experiments/exploits.
What are the chances of anything coming from Mars? A million to one, which is just a wild guess, has been said .......The Eve of the War
:-) This perhaps explains Microsoft's decision to funnel billions into OpenAI for its future software and why Elon Musk is so rambunctious
Education is a two-way process - real students ask questions, whereas the current crop of AIs don't, and you can tell a lot from what questions they ask. Perhaps this could be the basis of an AI-proof exam, where the question doesn't provide all the necessary information, and the student is expected to ask for it.
Yes, but that is probably an "easy" feature to build into it. All it needs is the real world data. Say, make a cheap, I mean hugely popular, education platform where students can ask their instructors questions about their assignments (on the same platform). We could call it Greedy Classroom or ItIsAlmostLikeLearning...
That's called "EdTech".. It's a huge industry of cloud providers all with very dubious privacy, which you cannot opt-out of. Much like you cannot opt-out of whatever slurpy tech your employer uses.
I found DJI's "RoboMaster" competition a little bit horrifying when seen through my cynical lens.. A bunch of kids write software on DJI's cloud platform to control toy robot tanks to battle eachother. I'm sure DJI would never ever do anything evil with that data.. right
Get the kids to teach the machine how to shoot, then tell the machine to shoot the kids.
Exams and related assessments have gotten more and more standardised and simple because the student:teacher ration is far higher than it used to be and lecturers either don't have the time, or can't be arsed, to assess students individually and properly.
Of course there is also the issue that tests are more standardised to remove personal bias for/against any individual students from the equation (at Uni, the secretariat coded each test paper to an individual so the lecturer only saw the code not the name, though I dare say that if a student wanted to make themselves known to a lecturer there were ways of doing so).
So, how to give a more 'personal' test while still anonymising the student???
Same here. Had a list of 20 odd candidates for 2 jobs. Boss selected four to interview all highly qualified - I selected three who seemed, from their CV, to have practical experience and a broad knowledge base but no relevant degree qualifications.
We interviewed all seven for the two jobs - the Boss agreed, after the first three of his selection, that they were hopeless! Two candidates from my list got the jobs.
Into what? A place where you just point a computer at a program that digests a load of text and can then spit out answers that regurgitate it, instead of learning anything yourself?
Fine. I suggest a test for all people overhyping this stuff. Train your "AI" on all the available material on aircraft design and ask it to design a passenger jet for you. Build the plane according to the exact specifications it spits out, with no human checks in the process. Get on the plane when it makes its maiden flight, across a large ocean.
That’s all very well, Howard, but some things just cannot be left to humans apparently, with the following an indicative tale ......
AI aids and abets with corruptions/alterations in free market trading ..... and indeed, it is the default primary leading lever to safeguard against unfavourable human corrections ........ https://www.zerohedge.com/markets/market-goes-haywire-dozens-nyse-trading-halts-open-after-technical-glitch
It’s handy that, isn’t it ...... having AIMachinery to blame or honour for human corruptions/alterations/corrections.
Crazy, eh? :-) Can you imagine where you would be without ITs autonomous help for selective input and/or reflective output?
“… An estimated 17 percent of students, […] said they had used ChatGPT to assist in fall quarter assignments and exams…”
Sounds like someone needs to tighten up the procedures for exam taking!! Do the exam invigilators not notice when someone is copying stuff from their phones???
And if these exams are being taken at home then ….well …that will need to change
Take home exams. And the main difference is that before moderately useful pseudo-AI you never knew if the take home exam/paper was produced by the student or their older sibling, parent, a broke graduate, etc. Now we have added one more option. Any exam not taken under strict supervision is just, at best, a training device.
It's not that hard for teachers to overcome ChatGPT in exams. It's to use the methods that have been around since Socrates.
All exams are done in person.
All exams are done by pen(cil) and paper.
No exams are taken out of the classroom.
All research papers must have verifiable references and sources (a lot of teachers will moan and groan at having to actually check the students/ applicants work).
Fail anyone who tries to use AI on an exam/application.
That's not true. A computer could pass mathematics exams decades ago. We still make people do them. Not because you'll frequently need to perform complex computations without a computer, which is why advanced tests now tend to allow a basic calculator to be used, but to verify that the student has the understanding of the concepts which are applicable in courses that cannot be automated or at least not yet.
Similarly, introductory computer science exams are trivially easy to solve with a computer, and nobody cares if you can pass one of them. It's still useful because it verifies that a student has the grounding necessary to take on more advanced courses, the solutions to which are not so easily automated. Other tests involve knowing a lot of facts that could easily be retrieved from a database or parsed from an encyclopedia, which are of use for future applications. If you were taking geography just to memorize where each country is, you could just as well look up a map, but if you're going to use that knowledge to do things like predict weather patterns, anticipate problems in international disputes, or arrange for transportation, knowing nothing and having to constantly refer to an atlas would make you much less efficient.
In some ways, it would be best if we could make exams a way for people to establish their skill level without having an effect on their credentials so there's no incentive to cheat. This isn't easy to do, though, since academic credentials have attained quite a large part in getting jobs and proving one's abilities. Condensing all of education into a small number of standardized tests could have other detrimental effects as well.
Knowing how to search for information online is a needed skill. Yet we don't let people use that skill to get out of having any real abilities. It may occasionally be necessary for your doctor to go to Google to solve a medical problem, but I think we can agree that someone who does that every time is probably doing something wrong and we wouldn't feel so confident if our airline pilot typed "How do you fly this aircraft again" into a chatbot or search engine when told they have permission to take off. I use web searches frequently, but it doesn't replace the skills I studied and can use even if the internet's down.
Basing your work off the work of others, regardless of how to obtained the work of others, has been going on forever. The difference between the information on the web and the information in a physical library is ease of access. Both require a student or researcher to use critical thinking to select and consume the right information.
I’ve played with Charcot and other ai sources to write software and it created a mess of version and release target incompatible code and used deprecated and obsolete language and syntax and required me to unpick and correct it so much I might as well have just typed it straight in with the aid of autocomplete.
Speaking of which, this little phone device chose to replace chatGPT with the incongruous word above.
I don’t think they just sit and do an exam. There must be several examinable components and some kind of dissertation-like project as is usual in PG qualifications.
I know a chap with an MBA, he’s as thick as planks and I’ve no idea how he dealt with the study. But he has absolutely amazing people skills and runs a very successful business.
Would be great to know what ChatGPT makes of the thousands of published & conflicting research papers.
We’ve already heard how many scientific published studies contain misleading information
https://en.wikipedia.org/wiki/List_of_scientific_misconduct_incidents
https://www.nature.com/articles/d41586-021-00733-5
Wondering if ChatGPT can parse evaluation and point out components that are misleading
[If such entities existed] should have some kind of mechanism where exam questions are detected and subtley erroneous answers emitted. So how does it detect exam questions? One way is for the exam setter to feed all the questions into ChatGPT as an "Exam Question Group". When someone feeds the same questions in as a user, ChatGPT weights the likelihood of them being exam questions depending on how many matches it detects. Ok, easy to game, but gaming the system does require some intelligence.
Did it not occur to the team that named it ChatGPT...
...what it sounds like in French?
What? Cat pee or number 2s?
Douglad Adams must be laughing wherever he is. The Sirius Cybernetics Corporation's genuine people personality equipped appliances are perilously close to avoidable reality.
Some hope though.
"The marketing division of the Sirius Cybernetics Corporation as 'a bunch of mindless jerks who were the first against the wall when the revolution came.'" (Encyclopedia Galactica.)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
