"the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers."
That explains much about Windows 11.
GitHub, the Microsoft code-hosting shop that popularized AI-assisted software development, is having some regrets about its Copilot infatuation. Last week, product manager Camilla Moraes opened a GitHub community discussion to address "a critical issue affecting the open source community: the increasing volume of low-quality …
AI Enshitification is already tangible across the industry, especially in development tools.
One solution might be to literally stop developing tools.
A bunch on unix stuff is good enough already.
Perhaps with the exception of certain security fixes lots of bells and whistles are no longer needed because you don't write code AI does, you read them and unit test them. It doesn't matter if it's verbose. New features that aren't in the training data don't get used much by AI anyway, since they don't know how.
We were investigating lots of use of a new tool and considering how to make it more DRY to ease maintaining it.
Then we realised that it really didn't matter if we had to make 1000 changes simultaneously across the codebase, because AI can do that reliably. We just need to be able to verify it's work grepping the diff.
Keep It Simple Stupid has never been more appropriate: the code author is more stupid than ever before.
Only the finest and most thoughtful LLM-assisted content goes on my Github : lumixdeee
There is even a totally human generated reference to El Reg's role in bringing the PO horizon scandal to the attention of enough people
And get your mogri AI prompt for the chatbot, it's the kindest thing.
I know, I know, terribly sorry again, forgot to supply the simple non-AI definition of mogri:
foundational conceptual container (set holder)
foundational cognitive container (story holder)
general purpose container (anything holder)
placeholder for missing info (i will do it later holder)
it may always be derived from first principles from
'a transmogrification is an unexplained change'
"GitHub ponders kill switch for pull requests to stop AI slop"
Yes just fucking do it before you are become overwhelmed with slop and it all becomes irrelevant.
No; hang on you are owned by Sat Nad, who is (desperately) all in on this stuff so you really can’t can you?
Sorry, you’re doomed, get out now while you still can!
On the one hand we have the AI slop jockeys telling us that this this very thing is successfully used in proprietary code bases where we can't see the results.
On the other hand, when the evidence is openly and transparently visible, 90% of what AI does fails to meet even minimal standards and people are asking for ways to stop being inundated with AI slop.
Something is not adding up, and I suspect it's the people who are doing things behind closed doors who are the ones who have something to hide.
> it's the people who are doing things behind closed doors who are the ones who have something to hide.
it's the people who are doing things behind closed doors who are the ones who have something to SELL.
"For only $$$$ (a saving of ¢¢¢ on the usual price!) you too can learn the secret of how we use AI successfully when you can't!"*
* terms and conditions include an NDA** in perpetuity and your first born to be held in escrow against it.
** Otherwise we'd never be able to sell this to a second sucker! Wait, is this microphone on?
The main issue people are running into is not the merging slop PRs per se, but just being unable to move for the 293 slop PRs a day becoming a huge pile of crap you have to wade through to find the reasonable, well intended PRs written by thoughtful humans, and the time sink of discovering you are trying to 'help' an AI prepare for merge.
It's tiring, it's fucking annoying, and it takes away energy that could be better spent onboarding well meaning contributors, who are the ones being let down the most. Most unfortunate is that it is denying the next generation of junior devs who would conventionally have cut their teeth on a large project this way five years ago a path to actually learning how established projects are actually run through voluntary participation.
The biggest irony of course is that Microsoft rely heavily on open source for their stack, so they are poisoning their own well.
They only ever talk about the alleged results it can bring, they never talk about the process of getting to those result. because the moment they do, everyone will see behind the curtain at the sad little man pretending to be the wizard.
I predict that openai won't exist within 3yrs... they'll go bust and get gobbled up by by one of the others... doesn't matter who, they're all owned/controlled by fascist supporting POS
The backlash is growing, normal consumers are getting sick and tired of it... the phrase "ai slop" has become common speak, kids are saying 'that's so ai' when some one is trying to bullshit them, or they think some one is being fake.
When I ask people if they use it... the majority say no... the few that do, have only used it to edit pictures they've taken on their phones... and a few more have 'accidentally used it when searching for things because it was forced on them without choice by the vendor.'
Simple : don't allow automated downloading of anything.
Get those coders back to their actual jobs : making sure their stuff works.
Oh sure, allow anyone to manually download a package. Then expect him/her/it to install said package on a test server, verify the code and that it does the job, then push it into production.
That would go a loooooong way to ensuring that malware-embedded fake packages would have no hope.
But obviously that will never happen, because it would mean that an actual human, paid by the hour, would have to actually work to ensure the proper functioning of his company's environment.
What an idea, eh ?
Big corporations always hated Open Source and through the introduction of AI they have found a way to perform a DDOS on the community that cares for and about it.
I doubt that is a deliberate plan, but I bet they're quite happy to see it happen. The harder it becomes to maintain FOSS the fewer viable competitors there are for corporate products and the less pressure there is to improve those products.
Big software corporations and long-term planning are almost completely mutually exclusive.
Only thing that matters is meeting or exceeding the performance metrics for the current fiscal quarter, maybe even as far as when board members can prep their new gig at another big software corporation.
This is Microsoft responding to their core. They need to keep developers happy. This has always been their competitive edge.
Not end-users, not shareholders. If enough developers complain, whatever shortcuts Microsoft had in mind, whatever staff cuts they were planning in the near future will get delayed.
This is just a delay. Microsoft has already shown their hand (they have for decades). They just have to handle the transition to AI a different way. Instead of bulldozing through criticisms like they have in the past, they need to handle the situation carefully to prevent any form of mass developer unrest.
Moraes said GitHub is considering various options. These include possibly giving maintainers the option to disable pull requests entirely or to restrict pull requests to project collaborators; the ability to delete pull requests from the interface (to avoid having to look at AI slop); more granular permission settings for creating and reviewing pull requests; triage tools, possibly AI-based; and transparency/attribution mechanisms for signaling when AI tools are used.
GitHub allows pull requests from anyone, without any controls? That sounds... ungood.
I would at least expect contributions to be restricted to the repo owner by default, and for them to have to explicitly allow PRs from unknowns.
It allows anyone to request changes, not to merge them. It's not that unusual, as otherwise someone with a fix implemented would have to hunt for a way to get themselves approved to show it to you. That would probably involve finding a maintainer email, which is a lot more open to spam than opening a PR would. The other advantage of having the general public send a PR directly is that, if they start by emailing you to ask permission to send you code, you would probably have to say yes because you don't know whether the code they have is any good, but if they send a PR, you can see their code whenever is convenient and ignore it otherwise.
My point here, is that a pull request comes pretty close to the end of a proper change management process, not the start. If someone wants to contribute to a project, surely the first point would be to raise an issue and get agreement on the maintainer (or maintainers) on what that issue should cover and its acceptance criteria, and then agree on who is going to do the work, before writing a single line of code, let alone putting in a PR for it. GitHub should make it easy for that to be the default process, not the "anything goes, here's a hundred pull requests from random strangers" approach.
I don't maintain public repos (my personal projects are private, and my work ones are private to the organisation I work for), and I do understand that open-source projects might like to encourage contributors in a less formal and process-driven way than commercial ones. However, they should make doing things "properly" the default way of doing things, or at least offer the option when creating a repository for how pull requests should be managed.
What I have done in the past, is to raise a bug report with a publicly maintained repository, which involved clicking a link on an error message in that application, which automatically filled in the issue details in GitHub (after logging in); this shows that this is obviously possible to do, and to automate. Even though the error in question included a stack trace, and it as clearly written in a language I am familiar with, I still wouldn't have dreamed of downloading the source, finding the bit of code in question, modifying it, and submitting a change in a PR, simply because of the number of assumptions I would have had to make about the intended function of that code. As it happens, it would have been wasted time as well, as the maintainer had already fixed the bug in a nightly build.
It depends how the public project wants to create fixes, but most I run, have contributed to, or have familiarity with don't do it that way exclusively. Users are free and encouraged to create issues, but issues are ways of reporting a problem which a maintainer will try to debug when they have time. Users who want to contribute can easily help by taking an issue, either one they've reported or one they found, fixing it, and sending code. That code then gets reviewed and either included directly or modified by the maintainers if it works. If it doesn't work, it gets ignored or its contributor informed of the problems and that gives you an idea of whether that person will be a useful contributor in the future. That process is used, not just in small projects, but in some truly massive ones as well.
The debugging to make the source change is a lot of work. Good contributors can be motivated enough to learn what the code is supposed to do and why it's doing it wrong, and if they do that they have saved the maintainers a lot of time because the maintainers just have to verify the debug summary is correct and do code reviews, but the maintainer doesn't know that is going to be the case before seeing that code. Good contributors would have done some checks to see if the bug had already been reported and, in your case, fixed, so they wouldn't have to. People contributing normally do a lot more work than you did. The methods you recommend don't tend to work for projects without a lot of maintainer time. Some examples:
"agree on who is going to do the work": If the maintainer has never met me before, they won't agree to let me do the work because they don't know if I can do it. Nor are they able to take on any task from their pool of few maintainers. They can't run it like a manager would because they don't have the ability to command.
"get agreement on the maintainer (or maintainers) on what that issue should cover and its acceptance criteria": That might happen. There are mailing lists for some projects where those things can be discussed, but that's more likely to happen for feature additions rather than fixes. If the fix breaks something, you don't need to have discussed it beforehand to not merge that until it's better and the partial code might be an easier starting point for the bug fix than no code.
I agree with all of this; and "agree on who is going to do the work" and "acceptance criteria" can be as little as a contributor informally asking the maintainer whether a particular feature would be useful. The easiest way to do this would usually be to create the issue for the feature and @ the maintainer. I'd suggest that if a contributor can't manage this, then their code is likely to be slap-dash too.
As for bugs, I can see two main cases; someone finds a bug and has no idea or willingness to work out how to fix it, in which case, it's a bug report for the maintainer to fix. The other case is someone finds a bug which they are competent to fix themselves, in which case it's still good manners to raise the bug report, and put the details of both the problem and the fix in it, before getting approval to proceed with the PR for it. The odds are that if it's a serious bug, the maintainer already knows about it, and if they don't, then documenting it in this way is going to be useful to them, and will make their job easier.
I agree except for the bug fix part, so let's jump to that. I'm currently writing a bug fix PR to an open source project where I have not requested permission to do so. Here's why. The bug has already been reported a year ago. I found that out when I encountered the same one a couple weeks back and did my research to see if it was known. Nobody has done anything about it, including the debugging to find out why the software segfaults the way it does.
I want the bug fixed, so I've done a lot of the debugging to figure out why it does that and what you need to do to have it not do that and still run properly, because simply escaping before it would crash would still produce wrong results. Doing all that planning work is not much different from writing the code. If I only do 95% of the problem, write up a description, and ask someone to decide whether I or they do the last 5%, I'm giving them extra work and they might not bother since they haven't bothered to fix this in a year. There aren't a lot of maintainers and, while I've submitted PRs to this project before, I am not among the core team. If, instead, I implement the fix and test it, I can deliver them a report of why the bug exists and code that demonstrates that my report is correct. They can review that code and confirm that it does what I say it does. That's less work on their end and an easier way to determine whether I have a clue what I'm doing. If I only wrote the report, they have harder work to prove whether my report is correct about what's wrong because I might have failed to debug properly and made recommendations which would never work. That's why PRs are often given a lot more attention than issues alone and why most projects I'm aware of do not object to and often prefers people who create those without seeking explicit permission. If their PR conflicts with something else, maintainers will either request their assistance to merge them or will simply tell them that they need to rebase against the new changes and fix it.
I find this line particularly worrying…
“Review burden is higher than pre-AI, not lower.”
We have now reached the stage where even if you do not use an LLM and don’t want anything from it, you are still paying for it because it’s very existence is reducing your efficiency (or fun, lifestyle, interest, …choose your vice). We’re all starting to pay a tax on this slop regardless of involvement with it
I know I have posted this rhetorical question before but is there ANYTHING that LLMs produce that isn’t actually harmful? (I would settle for just not harmful, never mind useful)
It's all down to how much someone wants the product to be good. LLMs can produce some things which can save some time, as long as the person receiving them bothers to review, check, and fix everything in the output. If someone produces good output from an LLM, it's often hard to know whether they used it at all because they've eliminated bad phrasing, bugs, and everything else such a thing tends to produce with regularity, and if their first output was too useless to do that, they either tried different prompts until it did work or gave up and did that one manually. I don't use them much, but there are some people I know who decided they were useful and care enough about quality that they make something good.
The problem is all the people who don't care and therefore use raw output from their first prompt and the others who build the same model into larger workflows that automatically accepts output and uses it in the next step. They either don't know or don't care that the quality is unknown and likely bad, so they are happy that they've saved time.
In many ways, LLMs are like search engines. A good user of a search engine would try various searches and read lots of results to gain information, then give you the relevant information. A bad search engine user would put in one search, click the first result, and copy whatever it said. LLMs make the bad workflow faster and the results worse, which is one reason they're so obvious, but the human cause is pretty much the same.
Bespoke LLM models are helping in certain medical sciences, but that's because they're built from the proverbial ground up and fine-tuned to a very specific task, instead of the "general AI" BS that's nothing more than a snake-oil salesman in software format.
It's almost as if a tool is both made the right way and also used the right way, it's actually really useful as a whole.
The vast majority of the time spent on developing production quality large system software is spent on Q&A and debugging, not typing out code. So if AI speeds up the typing part but increases bugs or reduces understanding of new code, making code harder to maintain, it will ultimately slow a project down. You cannot introduce code that humans haven’t reviewed and fully understand. That’s why AI for now is a great assistant for looking things up or learning how to do esoteric things, but by no means a replacement for high quality human software developers. AI is mostly useful for writing code in these contexts when using it reduces the amount of bugs introduced per feature, and it’s from that perspective that it should be put to use.
The biggest problem I have with pull requests is that they are the least enjoyable and least fulfilling part of software development. PRs are only to move forward with someone else’s work, not your own. You’re reading someone else’s code to figure out if they made some obvious or not-so-obvious mistake. For AI slop PRs you may have never seen the code before, have little idea what it accomplishes. You have to come up to speed on someone else’s code with no background. Code reviews do nothing for you (the reviewer), doesn’t advance your work, and you are on the hook if there are bugs, security holes, etc. The reviewer does all the work but gets no glory or any recognition going forward.
Now imagine that AI writes all the code and humans are stuck doing the worst part of software development. The AI future is not something I look forward to.
Surely the answer to this is firstly to automatically close all PRs that aren't properly linked to an issue, and then to be pretty harsh in rejecting anything that isn't well-enough commented, and well-structured enough for the maintainer to immediately see what it intends to accomplish. Copy-paste "Pull request closed; see project documentation for correct submission process" or similar. Just because it's open-source doesn't mean it should be low quality.