Well written
This is one major reason why I read El Reg regularly. A well-written, thoughtful, researched article that is not only topical but thought-provoking as well.
Bill Gates is worried about AI. Nobody’s preparing for a future of mass unemployment, inhumanly powerful cyber attacks, and a massive accumulation of power and wealth among emperors of tech. Yes, that Bill Gates. Really. A mere 25 years since the Redmond global octopus squeezed its tentacles tighter with Windows XP, yer man’s …
... Dr. Jason Gibson at Alcorn State University in Mississippi flunked 32 out of 35 students in their mid-terms for cheating with AI.
The fact that this happened clearly shows those 32 students' absolute lack of ethics, not to mention scruples.
To achieve any idea or concept of those qualities, they should have had a proper education and most of all, examples as a guide.
Evidently not the case.
This the numbers are statistically insignificant (?) within the US population, but 32/35 is absolutely mind numbing.
.
> 32/35 is absolutely mind numbing.
If the 35 were a truly random sample of the US population the result would still be significant. Of course it is nowhere near being a random sample.
You might have imagined that even these dumbarsed Americans might have read their illicitly AI generated essays before submitting. Although quite probably none of them knew enough about any industrial revolution to realise that Madagascar was unlikely in feature in one.
It isn't as though the US wasn't already scraping along bedrock dumb without needing to add AI to the shitshow.
What's a history professor doing teaching law students?
Wait you mean these weren't law students? Wow I'm flabbergasted. I didn't think any other course required the students moral compass to be surgically removed upon entry. Well, other than maybe the MBA students...
Good observation.
The mention of AI affecting learning in IT is just one aspect.
Main stream Education has been sliding into mediocrity for the last 20 years. Parenting by many has also become optional. And now AI has discouraging youngsters from even thinking.
This will cause the majority of society to be selfish idiots and that will lead to severe consequences for all but the very wealthy. :(
This was a midterm exam about the industrial revolution. Either a required course or one of the options to meet the requirements. If you have a lot of people who see a hurdle in front of them a way to go around it instead of jumping over it, many will go around it. This is the modern equivalent of using Cliff's Notes instead of reading the material, which I'll bet a lot of people reading this did for classes they felt were pointless requirements or otherwise keeping them from spending more time on the classes they cared about.
If this was happening in a class of 35 students studying a subject they are interested in and motivated to learn I'd be way more concerned. But college has a lot of "required course" type hurdles put in place that many students view as a necessary evil to get to what they care about, the courses in their major. Now yes there are students who view the courses in their major as a necessary evil, because they aren't interested in college at all - they are interested in the degree that will (they hope) give them better job prospects.
That's kind of the fault of modern corporate hiring practices where "a college degree" became necessary for a lot of jobs where it makes little difference. If you are hiring an engineer or a nurse you want them to be educated in that subject. There are a lot of jobs where it makes little difference - at least there is no board exam to be certified as a computer programmer or working in HR so requiring degrees for those jobs creates the sort "going around the hurdles" behavior here.
I agree with many of your core points, but disagree strongly with some others.
"This is the modern equivalent of using Cliff's Notes instead of reading the material, which I'll bet a lot of people reading this did for classes they felt were pointless requirements": No, it's not. Reading short notes, or Wikipedia, or having an LLM write a summary for you to read, all result in you learning at least a little. Then you would do some actual writing. It wouldn't be as good as reading the full materials, but you still do very similar work. No, there was an analogous thing that people used to do for this: buying a paper from someone else, whether it be last year's, paying a friend to do it for you, or online essay mills. I didn't in fact do that, even when I didn't give the work the attention it deserved, and those who did were rightly punished for cheating.
It is correct that a lot of people will go around hurdles, but that doesn't make a case that we shouldn't mind when they do it. I consider it a good thing that we don't have a licensing test to be a computer programmer, because I expect the people who tried to set up the test wouldn't be great on quality or ethics*, but that doesn't mean that people should ignore what they learned. Hopefully, they just go to their first interview, fail humiliatingly, and go back and learn something, but if they don't, we all have to live with the result. I'd consider the argument that people shouldn't need to take this course. I would not consider letting people off because they decided on their own not to care about the thing they agreed to.
* Programming in general is so broad that a licensing test for all of it would be either very easy or too focused on trivia. I would be concerned that a lot of it would become focused on specific languages or tools, either because that's what the setters are familiar with or because that's what the corporate sponsors wanted to get included (we'll be running Oracle databases on Google Cloud for this test and you must write in NodeJS). Ethically, I think a licensing test should only serve to determine competence, not, as some do, to restrict the supply of licensed people or to collect a lot of certification and recertification fees because you have to pay them if you want a job. This is why I'm happy there isn't one.
I'm just a computer guy, not an auditor or lawyer, so a lot of this stupid logging and paralyzing security is getting in my way.
Which is to say, one man's unnecessary hurdle is another man's compliance with GDPR. Which is to say grown-ups sometimes have to do things that they don't see as valuable for reasons that are not theirs to choose.
Well, there's several ways this student could have smelled the Madagascar rat in this one, but they do suggest better due diligence from her than from the remainder of her cohort, imho. I have to think Jason Gibson didn't just go 'oh dark mode, woowee, points back!' like a wide-eyed squirrel, but rather questioned the student, checked it out, verified the work, and then decided to restore her grade according to the results of that.
And speaking of white text fonts, you can see an example here at ElReg under the Skype-equette heading. The hidden text shows up when you select it with the mouse ... (font size should be changed too for better stealth). So I imagine plenty of ways students could have visualized hidden text here too. Also, if this student had an LLM do her homework, it seems plausible at least that she proofread its output (as recommended), saw the Madagacar bit and wondered why it was there, and then maybe investigated further and found the hidden prompt via dark mode(?).
Still, yes, as shown also by the Roberto Serrano affair at Brown, the majority of students use AI (so-called) for homework and exams whenever feasible, as they "choose to become idiots" for our species' collective future benefit (</sarc>).
Students used to have to work really hard at figuring out creative ways of cheating without being caught, and those life-building skills became essential when they later engaged in their work life, their trade, their interpersonal exchanges and all. I expect technical and cognitive debts to grow astronomically now that cheating has become a mundane and frictionless pushbutton affair. The deskill will be catastrophic ... I mean, we used to have folks who were either good at doing stuff or good at cheating. Soon we'll have folks who are good for nothing, trained properly at neither, void main(); ... ;(
I still think the most plausible explanation is that she got a failing grade, found out why, came up with an excuse, and since nobody can disprove that excuse, they felt they had to grade her essay after all. The alternative that she generated an essay with an LLM, read it, noticed the unusual text, went to the prompt to understand why, found the prompt salting with intentional colors to hide it, then decided it would be a good thing to leave in, doesn't strike me as likely.
Her story falls between the two. I suppose I could see a student reading a nonsense instruction and deciding that, stupid or not, it was what the assignment said so they'd carry it out. The old and often used story of a test whose instructions tell people not to answer the questions is one a lot of people will have heard, so I suppose someone could think there was a trick like that going on. I remember a couple similar tricks in my own education, though usually hiding extra credit opportunities which you would have to find rather than the other way around. However, with something that stupid, I'd expect that most students who found it would ask for clarification unless they had procrastinated and needed to complete the assignment at the last minute, and she didn't do that, hence I still think it's more likely a successful lie.
Yeah, different strokes I guess ... Speaking of which, watching the coverage of Sophie Adenot in space and her mentions of 'courbatures' and 'stylo-bille' made me wonder: could requiring homework and exam answers to be handwritten in cursive (yes, by hand) help with this right curse of instant LLM-produced slightly randomized artificial submissions?
It'd mean a student would have to manually re-copy the evil-software's otherwise malnutritious excretions (if used), reading it with at least one eyeball, and in so-doing might actually learn a little something that would be otherwise lost, plus develop "increased connectivity in the brain" if any, for healthier overall operation (including that related to irreverence).
Maybe that could help some. Inquisition minds ... ;)
I doubt it. Nothing says that writing something makes you understand it. People can copy stuff in languages they don't speak as long as they can write the letters well enough. All that does is drill people in cursive handwriting. The report about brain connections doesn't prove those connections have a benefit. For all we and they know, those increased connections are just what you need to write by hand as opposed to typing. We could hope that someone will bother to read what they're copying, but I think we're grasping at straws if we go that far.
I think the only answer is revolution on multiple levels, a la French model first, then on the Russian model, then on the Tooting and Judean models. Billionaires are the new aristocracy, guillotine the lot, collectivise all private property, then all meet down the pub to discuss how to get rid of the Romans.
Capital punishment for anyone caught littering, walking on the cracks in the pavement or wearing a loud shirt after the hours of darkness.
I think that's quite reasonable, don't you?
* The majority of AI hacking is in bypassing its security and guardrails.
* AI system output will not be new or creative; it can only spit out 'that which has come before' (its training data), or combinations/variations thereof.
* AI output is irreproducible and indeterminate.
* AI has great appeal to the masses, because you "command" it in your native language, versus traditional computer languages. You don't have to know about algorithms, data structures, etc. to get "an" answer.
* AI has great appeal to the masses as an oracle. Easy answers, just ask AI. It's the new religion.
I had a college-degreed friend telling me about her use of AI to do numeric calculations to produce a tide table. She blithely told me, "Oh, I just tell it not to hallucinate."
People these days just want answers -- any answers, and truth seems no longer to be a requirement of those answers.
> truth seems no longer to be a requirement of those answers.
I am building quite a list of 'you are right; that is completely wrong, I made that up' admissions from AI. Sometimes because I know the subject area, sometimes because what it claims is inconsistent with itself, sometimes because I simply don't believe it and ask for clarification.
I asked it about myself and the bullshit it came out with was quite impressive, full of parse and undeserved credit, which has no basis in reality.
It is what we should expect from something trained on bullshit and misinterpreting facts.
I teach a writing course for STEM students. Many start off thinking they can't write and so some will try to get an LLM to do the work.
So I have face-to-face check ins. They have to explain the topic and sources. They pick their topics so they are invested. Then I show them how to use LLMs for research and topic development and how to be skeptical of AI output
Instructors that forbid students to use LLMs and set out to catch them aren't doing students any favors. These tools exist and they will be required in many workplaces to use them. So I treat it as an extension of search engines, spell check, Grammarly, and other existing writing tools. Because if I don't teach it Reddit will.
That is your opinion, but there are a lot of tools that exist and will be expected in workplaces that, for various good reasons, instructors exclude when teaching. For example, when I taught assembly, we made people handle a stripped binary. There are tools that produce pseudo-C from those. If I had to reverse engineer something, I would use that. They didn't get to, because the goal was not the successful understanding of what the code did; we could easily answer it as we had the source already. The goal was for them to understand the assembly that a compiler wrote so they could do the next assignment where they wrote some assembly by hand. In turn, that was done not because we needed that application but so they would understand how their code interacted with the hardware and how performance and resource usage were affected. Unless they ended up writing compilers, extremely performance-sensitive code, or something OS-ish, they probably didn't write assembly again, but learning it was more important than all the tools they might use later.
I wonder what skills you are supposed to teach and whether your LLM-centric approach is actually doing it as well as the previous method would be. A lot of the writing that STEM students have to do isn't that focused on everything being grammatically correct, but it is very focused on everything being technically correct, clear, and ideally well-structured. When they have to write to explain a complex thing they've built, are you confident they'll have those skills?
First thing I would teach is reading comprehension.
I do not have an LLM centric approach. I have an approach that requires the writer to make intentional choices about their writing.
I have expanded my classes on information literacy to discuss the problems that LLMs present and the importance of running down original sources and reading the information in context. You may notice on a reread of my post that I discuss using LLMs as A TOOL for RESEARCH AND TOPIC DEVELOPMENT. Also that I have face to face meetings to discuss their research. These are tools that have specific uses. AIs do tasks, not jobs. I teach my students to use tools to do their work better, not to do it for them. Every writer needs to develop their own voice, and no LLM can do that for you.
I have been doing this for a very long time. I remember when Publisher came out and any idiot could make flyers and brochures without an exacto knife and paste. But it turned out those idiots still didn't know design and their work was cheaper but not nearly as good as what I was doing. The tool is just a tool. It does not replace experience and skill.
So I would be interested if you have anything to say ABOUT THE POST I ACTUALLY WROTE.
I had plenty to say about the post you actually wrote. I don't think you agree, hence your attempts to claim that what you said contradicted me. It didn't.
LLMs could be used for research, topic development, or writing. From your comments, we know your students at times use them for all three. From your comment, we know that you permit that, though you try to instruct them where doing that makes sense and where it doesn't. We can assume you know what you're talking about when you do that.
I think there's reason for concern in all three of those, But I mostly spoke of the one I consider most concerning, using them to write. Once again, you made it clear that you allow that and feel we should: "I treat it as an extension of search engines, spell check, Grammarly, and other existing writing tools". Your opinion is that instructors should not forbid LLM use. My opinion is that there are plenty of valid reasons for them to do so even if LLM usage later would be acceptable, and they should consider for each assignment whether forbidding or allowing LLMs is the right approach, perhaps combining some teaching of how to check for hallucinations while having another situation where they need to write something unaided. I do not think face-to-face discussions fix that problem, so my failure to credit you for that was not because I didn't notice them, but because I don't think it matters to the discussion. I think using LLMs as a tool for topic development is risky because it's one of the things STEM students will need most in their writing and in situations where LLMs are worse at it, and I did mention that.
It's clear we disagree. I don't think it helps when you insinuate that we disagree because I failed to find the magic sentence that made your arguments clearly right. We disagree because I think the things you do, which are mostly good, are not enough.
I never once said that I allow students to use LLMs to write. I explicitly forbid using LLMs to do any writing turned on for grading. If you can point out where I did say that I will say you are hallucinating like CoPilot.
Describing my approach as "LLM centric" is quite frankly baffling. Saying a tool exists and there are hood and bad ways of using that tool is just reality. We spend far more time on writing concepts and conventions without mentioning LLMs at all.
"I never once said that I allow students to use LLMs to write."
I suppose you didn't, but you did say "I treat it as an extension of search engines, spell check, Grammarly, and other existing writing tools". But fine, let's assume that there is an obvious difference between an extended writing tool and a thing used to write and chalk it up to my stupidity. A writing teacher would know how to express that clearly, especially in a discussion about when it's appropriate to restrict how students write with LLMs, so if I didn't understand, it must be my fault. I also have concerns about what "topic development" means because my reading of it suggests that it's one of the things I really think the students should do without LLMs, but getting a clear definition of that would probably take three more posts so let's skip it.
"Describing my approach as "LLM centric" is quite frankly baffling."
You include specific lessons on LLM use. Whether by your choice or the institutional requirement you admitted, that's a part of your class. And to be clear, I don't mind if that's what you choose to teach. I just wouldn't want to take that class. However, I disagree with your original statement that "Instructors that forbid students to use LLMs and set out to catch them aren't doing students any favors". I think they are, and that's the main point I've been making.
Rather than consider my disagreements on that, you've chosen to quibble over words such as "centric" and throw doubt on what you even do, because that statement advises against forbidding LLM use and this post says you do forbid LLM use, at least in one area. Worse still, the example that prompted you to make your original statement forbade and caught students using LLMs to ... write, exactly the same use for which you're now claiming you don't allow them either. Suffice it to say that I'm entirely unconvinced.
The issue isn't that AI will stop people for looking under the hood (though it will, eventually, when people start taking AI's word for what's under the hood), the issue is how those eager students of the dark arts will be able to earn a living with what they've learned.
It doesn't take a great deal of imagination to reason that digital banditry will be a growth industry. Everyone else will employ AI to do their coding and software design.
I don't know if an LLM could do it, but the problem is that a human could:
"While most summaries of the industrial revolution speak of it as a predominantly European and North American affair, this is not an accurate description. Even countries as small, remote, and impoverished as Madagascar successfully industrialized. By the 1840s, half a century before being colonized, Malagasy industry was producing modern weaponry and consumer goods, mostly for domestic consumption. Madagascar was less connected to international trade networks and had much less individual wealth, so its industrial base did not grow as quickly in the late 1800s."
So before knowing whether students were cheating, it would have been dangerous to judge them on that. They already had trouble sustaining the judgement when a student claimed she followed instructions, even though those instructions were stupid and it's implausible anyone would. It wouldn't have helped to give them a more plausible instruction to hide behind.
Too few of the people doing the teaching are giving the learners guidance on how best to use AI. Mostly because they've not yet worked it out for themselves. Given the speed at which the technology is moving, getting in front of it will be hard enough if you know you need to get in front of it; somewhat harder for everyone else.
This was called the "Singularity" by the technical hoi-polloi. It was predicted to arrive to arrive around 2026 by some, which I thought was a little soon.
I guess I was wrong?
And is it really anything new?
When the printing press made it possible to disseminate information at a faster clip, with less centralized gatekeeping, it was hard to predict what would happen next
When mechanization started reducing the need for farm labor. this also represented an event where it was hard to predict what would happen next
When the assembly line started reducing the need for skilled factory labor. again there was a period where it was hard to predict what would happen next
When the computer started reducing the need for manual labor around brute force mathematical calculation and data collection it was also hard to predict what would happen next
When the internet made it easier to disseminate information around the world, again with less centralized gatekeeping again it was hard at the time to predict what would happen next. And we are still not done with that particular revolution and yet here comes the next one!
Mabey the rise of generative machine learning and the large language model (AI) feels scarier because it can be more generically applied. But all the above eventually led to the ability to move more work into the digital realm. So its not surprising that the next big thing would apply more broadly to the world.
Just like with the printing press the tools for doing something relatively complicated have become more accessible to many. But the complexity remains in place. Its just that more complex things can be done by fewer people and its becoming harder for those people themselves to completely internalize and understand all that complexity without the aid of the new technology itself and the will to ask the right questions.
This is not the Singularity you are looking for. Move along.
SERIOUSLY, the singularity is either a compute system capable of designing a complete system superior to itself, or, more recently, a fully-self-aware system. None of these are dreaming, let along dreaming of such accomplishments.
Presented as Gates does, it's like the asteroid mining thing (we fund their mining because they contain rare metals, which then become common; the price collapses and it's no longer economic). If huge numbers are out of work (which tbh I doubt, but anyway), who will buy all the services and stuff that keeps the economy ticking over? UBI will only go so far (though I'm looking forward to getting some cool kneepads).
I'm with Adama tbh. Keep em in their boxes doing useful stuff. And branch tf out from using gigantic LLMs for every ML application. That's how it'll go in the end anyway. Especially once use is charged at cost.