Posts by Conor Stewart

Anthropic wants Claude to play with money, unleashes finance agents

Fundamentally wrong approach

Current "AI" is little more than predictive modelling, it has no ability to think or understand. It can be good for spitting out responses based on its training data but that is all it is really good for.

Why are they attempting to combine extreme predictive text and finance? The LLMs that exist currently have a fundamental flaw that impacts any kind of decision making or creation and that is that they cannot think, reason or understand. We need an AI that can actually do these things before it will be any good at tasks like this.

That is just about LLMs though, simpler models have been able to successfully trade stocks just based on pattern recognition and forecasting but they are much more specialised and can't do much other than trade stocks. That is an example of a good use of machine learning that actually works, just like reinforcement learning in robotics and classification of data sets or object detection.

Current AI is only actually that good for small specialised tasks and unlike LLMs they don't need huge resources to train and use. LLMs are just fundamentally the wrong approach and everyone just seems to be trying to throw more compute and training data at it.

Yet another experiment proves it's too damn simple to poison large language models

Re: What happens to knowledge...

Just look through history, loads of examples of wrong things people believed because someone they trusted told them. Quite commonly they become things that society just "knows". A lot of it in the past has come about due to religion and likely from the intention to deceive and control people, or sometimes it is just from someone guessing or making theories and people just take it as fact.

This is something that has continued and grown with the internet, too many people just accept what they see online, now it is just extending to LLMs which have no ability to tell if a source is reliable or not. This has always been a problem in various forms, LLMs are just the most recent evolution of it.

"If it's that unimportant, then what's the reason to lie?", people lie about unimportant things all the time.

The point of this experiment was just to show that it can be done and how easy it can be done. This was a single website and Wikipedia article. What if instead of something unimportant like this there were a load of fake news sites created just to spread misinformation and targeted specifically towards LLMs. In this case because there were no other sources it only required one fake source, the more sources the harder it is to fake, in theory since LLMs don't apply any reasoning all it takes is for them to retrieve the wrong website, it's not like they compare and cross reference sources. Whilst a human may be able to look at a website and think it is dodgy, an LLM can't do that.

Add in the fact that LLMs can get results from social media and it makes it much worse considering the amount of misinformation on there, often spread deliberately.

Just look at the existing cases of LLMs misinterpreting or trusting sources they shouldn't. They have even been found taking satirical responses from Reddit and presenting them as facts, no cross checking or anything. So all it really takes is one bad source that the LLM happens to use and it is wrong, the higher the ratio of bad sources to good sources and it becomes even more likely to respond incorrectly.

The whole name "Artificial Intelligence" is just incorrect, there is no intelligence going on, it is just a fancy model for predicting responses.

What is concerning is the amount of trust people put in it. At least if you know the topic yourself you can notice when and where it might be going wrong but if you are just asking it about something you don't know then you have a lot less context and knowledge to tell if it is wrong without going and doing research yourself which kind of defeats the point of using LLMs.

GitHub opts all CLI users into telemetry collection whether they want it or not

Seems to be a bit of a contradiction

>"If you want to see exactly what would be sent without actually sending it, you can enable logging," the help page reads before providing a sample of what might be in there. The sample payload includes an agent field, architecture, device ID, OS, flags, command name, invocation ID, and other metadata. Actual telemetry payloads, GitHub noted, may differ considerably.

So you can see what exactly would be sent by enabling logging but actual telemetry payloads may differ considerably? So really the sample payload is false and misleading and likely just meant to make people think it isn't that bad?

Bcachefs creator insists his custom LLM is female and 'fully conscious'

The answer is in how these LLMs are designed and trained. They are fed lots of text and they essentially find patterns in it to predict answers to questions. Everything it can output is part of what it was trained on, except when it hallucinates.

It is incapable of coming up with a unique idea on its own other than shoving random concepts together likely in a way that has been done before.

Looking at using an LLM for programming, it is only capable of writing code similar to what it is trained on anything slightly unusual or less common and it usually fails. This is because it can't think and doesn't have any understanding of code, it is just predicting based on its training data what the code should be.

Re: Scary, truly scary

No, reality is reality, language is just used to describe it.

Rust Foundation tries to stop maintainers corroding

Re: Something isn't right

Again like I said it is an issue with the project, especially if they make it difficult to say no.

My argument is that accepting funding from these companies instead also gives them more power and leverage over the project and the ability to push the project in the direction they want it. It would be quite easy for a company to put a lot of funding into a project, like enough to hire full time Devs and then threaten to pull funding if the project doesn't do what they want, unless the project maintainers are very strict about not tolerating this and are fine with firing people because they no longer have the funding then they will get pushed around by large companies.

Yes a team of Devs working for a company would only make improvements that benefit the company, but it is the job of the project leads to make sure that they aren't deliberately hampering other companies or expanding the project beyond what it is supposed to be. Also competing companies can all hire their own teams if they want, then they can all improve the features they want, that way it works out pretty fair. I also don't believe that changes made by one company that are allowed into the project because they align with the project direction would be of no use to anyone else, so even if the company is only doing things that benefit them they won't just be benefiting themselves.

Re: Something isn't right

No one is forcing them to work on rust, it is voluntary. If they don't have the time to work on it without getting burnout then they shouldn't do it or should reduce how much they do. It is just like having any other job and then doing it as volunteer work too on the side.

I don't have much sympathy for burnout when it is self inflicted, especially when they are doing it for free. Yes if you are working a job to earn money for living expenses then that is different, you may not be able to avoid burnout but doing your day job voluntarily in your free time isn't the same. You could just do less.

I am not against funding and full time open source developers it just needs to be managed very carefully or it is pretty likely to end up making the problems they are aiming to fix worse, particularly burnout or getting pressured by companies. Then there is also job security. I don't see anyway in this so far where there is even a tiny amount of job security. Then there is also the issue of how funds are distributed, who gets to decide? I can see this being a very bad thing for open source, people getting exploited or left out. Exploited basically like a pyramid scheme where there is the promise of pay and possibly full time work if you show you are good enough by contributing or the funds only going to help out the head of the project and their friends.

In my mind a better option would potentially be for companies to hire their own Devs and get them to work on the open source projects the company relies on, that way they are full salaried developers and it keeps the project mostly clear of the issues that would come with introducing money into open source.

Something isn't right

People don't tend to join open source projects to make money, they join because it is something they are interested in and want to contribute to. Paying them isn't going to magically solve issues like burnout or pressure from people using the project. It may even make it worse, instead of being something optional for them to work on now they are required to work on it or not get paid, especially if they left their job to become a full time rust maintainer. Almost sounds like it is turning into a company but without any of the responsibilities of being a company and employing people.

Anyway my main point is that it won't fix the issues mentioned, instead of having burnout they will now have burnout and money. It's basically like if a company is overworking it's employees but says they will give them a raise so it is all fine, the raise doesn't stop the employees being overworked. Large investors will now have even more leverage to apply pressure to move the project in their direction, basically they can say "do this or we won't fund you any more".

Yes open source could be funded but it needs to be managed well and it's goals realistic, not whatever this is. The problems they mentioned likely come from within the project or in the case of pressure from users that just comes with being an open source project, not even just open source though, any time you make something to be used by others you will have an amount of feedback and demands to improve it, that is just what happens when dealing with people.

Python Foundation goes ride or DEI, rejects government grant with strings attached

I don't agree. There are plenty of situations where it would be acceptable to accept a grant with strings attached. Like if the grant required certain features to be added or improved, as long as those improvements are still open source and available to all then why would it be an issue? It is up to the maintainers to decide if the proposed changes and improvements align with the direction of the software and if they want to accept the funding to add those features.

You can't have it both ways, wanting open source to be funded by the companies that use it and the companies having no say in how the funds are used. If the companies are getting no support or cooperation from the maintainers to improve areas of the code that are of concern or to add features that would benefit everyone then they wouldn't keep funding it. You wouldn't keep funding something if they completely ignored your requests and just did what they want.

Imagine you are a company, you have found an issue in the code that only affects a few edge cases like yours, you can either wait for the maintainers to fix this issue which isn't a priority or you can essentially pay them to fix it. If the attitude of the maintainers was that since it doesn't affect many people they don't care and won't fix it any time soon then you are unlikely to stay with them and keep funding them. However if they let you give them the funding they need to keep the project going and potentially hire more full time developers then why shouldn't they work on that companies requests.

One long sentence is all it takes to make LLMs misbehave

Re: Here's a suggestion

That is only true in principle and likely easy to get around. For the LLM to have no data on how to make a bomb it would have to not have data on a lot of chemistry and physics. Sure it may end up without any data on bombs themselves but it will still have data on chemicals and their properties (like explosiveness), it could still likely explain an ignition mechanism too. You may have to word questions in a chemistry or engineering context rather than asking about bomb making but I can't see how just excluding bombs from the training data would help.

To be truly able to exclude harmful things like bombs then the "AI" would need to have actual intelligence and understanding and would need to know what a bomb is and how it is harmful

.

How does that have anything to do with what is happening here? You seem to have a major misunderstanding of how LLMs work.

Problem PC had graybeards stumped until trainee rummaged through trash

That is a lot of assumptions that don't hold very well. They say they used the PC for training courses and things like that? Go into nearly any office and you will find soda and candy. It is far from unusual for someone to have a drink and eat sweets when working on a computer.

As ChatGPT scores B- in engineering, professors scramble to update courses

Re: But when deeper thought was required, ChatGPT fared poorly.

It doesn't even do that. We don't really understand how thinking works so we can't replicate it. These models show almost no ability to reason so they likely can't and just predict what the answer could be given its training data.

Devs sound alarm after Microsoft subtracts C/C++ extension from VS Code forks

Re: Shouldn't have been unexpectected.

Microsoft is just being Microsoft but this is cursors fault too. If Microsoft have in their license agreement that the extensions can only be used with VSCode then why were cursor finding workarounds and ways to hide that it was their IDE accessing the marketplace? Why are they still looking for workarounds?

They knew that what they were doing was against the license but in typical AI company fashion they don't care about licensing or copyright, they just do what they want. So yeah what Microsoft did isn't great but it was there in the license all along, it is just them enforcing it now. What cursor did however is to ignore the license and just do what they want whilst trying to hide that it was them and they are continuing to try and find ways to violate the license. I have no sympathy for cursor here.

Fining Big Tech isn't working. Make them give away illegally trained LLMs as public domain

Exactly, it isn't like you could host it effectively on your home computer. Their business model wouldn't even change much, they would still charge for subscriptions to their cloud service where they host the model, it just means any other company can do the same using their data centres.

Re: Not a Remedy

Exactly, if the issue is copyright then this solution is the equivalent of someone copying your music or book and selling it and then being made to release it for free, yes it hurts the person infringing copyright but it also greatly harms the copyright owner.

Re: Are you new here?

If you read a book then someone has likely paid for that book. If you read information on a website then either it is provided for free, paid for by you or monetised through adverts.

If a LLM uses information gathered from advert monetised websites then how much money does the website make? Only the money from the initial harvesting from the LLM which will be equivalent to one visitor and that is only if the LLM loads the adverts. The website would lose out on a lot of money if everyone accesses it's information through the LLM instead.

People pay for what they consume one way or another, even if not obvious or indirectly and the owner sets the price they want for it. There is no need for a portion of profits or salaries because the information is paid for or free.

Your argument is based on incorrect assumptions.

You say you are more concerned about privacy than anything else, yet you aren't. If you were really concerned with privacy then you would be advocating for the deletion of these models and training data and nothing less. Instead you are concerned about the environmental impact and wanting the companies to not profit off of it. There have already been exploits to make LLMs output things they shouldn't, do you really think that the companies will continue patching these exploits if they have to make it public? Even if there is currently no way to extract personal information once the model is public it would be available forever giving people more than enough time to find ways to extract data or just to keep it archived until the technology required to do it is available.

Re: You are an LLM

Human brains are in no way comparable to LLMs. Human brains function very differently and have different functions like being able to understand and reason and have unique thoughts. LLMs are just pattern recognition. We don't even fully understand how brains work (if we did then making a copy of one would be easy) so how can you claim that brains are just LLMs?

Re: Potentially another option - poison the well?

Maybe you could do it with images or audio too, similar to how people hide data in the least significant bit of images (stenography). Maybe if that was done enough (maybe the 2 or 3 least significant bits) and the model is sensitive enough it could mess with the model.

Re: Imaginary Property does not exist

It is punishment to the company but it is also punishment to the people who's rights are violated. It would be like if someone copied your published music or copied parts of your book and instead of being made to delete it they are just made to make it available for free. Yes it punishes the company for copying your work but it also punishes you for no reason and to most people wouldn't be an acceptable solution.

Re: Garbage in, garbage out

On top of that there has been research to suggest that better preparing and selecting training data can lead to smaller models and better results. An example I read about was about a model for working with images, I can't remember what exactly it could do but they generated training data by paying people to look at an image and generate a description, they would speak their description and it would be captured by speech recognition software. They got the workers to describe the image in quite a lot of detail. After training the model was smaller and better than other models that were trained on a lot more data, like the whole Internet.

One problem with the current approach to AI is that it has no understanding or reasoning, it is all just pattern recognition and advanced predictive text. It has no way to tell if the training data or it's responses make sense, this is why training data quality is so important. I think that the issue of LLMs hallucinating is because of how much nonsense data it is trained on, anyone who has spent time on the internet knows that it is full of incorrect information even amongst otherwise correct information. If an AI is being trained on this huge mix of correct and incorrect and contradictory information with no ability to understand or reason then of course it will get confused and generate nonsense or incorrect outputs.

Until AI has the ability to understand and reason it will have issues with hallucinating but this can be partially avoided by being very careful what you train it on. If AI could understand and reason then it could weed out the incorrect information and check what it is outputting makes sense and is correct. However we are a long way away from AI being able to understand or reason, it is a massive jump from current "AI" technology and what we have now is basically decades old ideas (like neural networks) taken to the extreme.

All universities have different arrangements. The one I went to states that students retain ownership of all their coursework and research because they are not employees so their IP does not automatically become the universities.

For employees it is different, the university owns the IP if it was created during their normal duties or if it was created using university resources. Research is either subject to those rules or can have its own arrangements. Externally funded research is different still and has individual arrangements between the researcher, company and university.

Even in the cases where the university owns the IP the inventor still has a part in it, they still receive a portion of the profits and still have some rights. The same applies to spin off companies, they need to pay the university a portion of their income.

If any third party can just copy someone else's research then companies will just stop researching. Say a company spends millions researching a drug and then after all that research another company just comes along and makes it themselves and sells it at half the price, what do you think happens to the company that developed the drug? They can't compete, they make no profit, they spent lots of money on development so the company either or goes bankrupt or stops researching, either way the research stops. Why then would companies research if they know they will just get copied and undercut immediately and the whole process will just lose them money? This doesn't lead to society improving long term. It may be good for some people short term but long term it just means no more research.

Your comment about extending IP protection to AI generated IP is just not a good idea in any way. LLMs are trained on data gathered without permission and everything they generate is derivative of its training data, there is nothing truly new or unique generated.

I am not the person you replied to but whilst we do need a patent and IP protection system it needs to change. In a lot of cases now it is abused and used to stifle innovation, both with companies patent trolling and patents being too long and too broad.

In my opinion patents need to exist but need to not cover entire concepts and abstract ideas, instead they should only cover implementations and only if the company actually uses the patent. For technology now 20+ year patents on whole concepts is too much.

As an example for pharmaceutical companies, a patent on a specific drug is absolutely fine, even long term, it is a specific implementation and other companies would only copy it, not innovate on it. However patenting a whole class of drugs or concept is different and should not be allowed in my opinion because even though the company may make drugs within that class of drugs it slows down innovation and possibly prevents good treatments from being found sooner.

Another example is 3D printing. The entire industry was held back due to 20+ year patents by stratasys on entire concepts.

In my opinion patents should be used to prevent people from just copying implementations, not for preventing innovation.

Re: Alternative solution

If you believe in your idea then go and do it and prove it instead of just commenting on how it would be so much better with no proof.

Re: Cui bono

Pretty much, just like they were with crypto mining. They had no direct hand in crypto mining they just built and sold hardware that could be used for crypto mining. They made a huge profit that didn't come from crypto mining themselves. So Nvidia is very much like equipment manufacturers in the gold rush.

It reminds me of parking tickets. If you are rich enough then you just don't care, paying the tickets is not a problem and they wouldn't even notice the difference in their bank account, they don't even pay them themselves, they just tell their assistant to pay them. They know they will get a parking ticket but there are no real consequences so they don't care.

Fines for large companies are just the same, the fines make no financial impact or have any consequences so the companies do not care. Or they use their expensive legal team to get out of it.

Re: Throwing the baby out with the bathwater

All this comment shows is that you don't have as much understanding as you think you do.

Re: Throwing the baby out with the bathwater

Just because it benefits you that doesn't mean it should be allowed or legal.

It is theft, plain and simple, if you applied your logic to anything else you probably wouldn't agree with it then. Is stealing money from people fine if it is given to charity or to poor people? Is stealing houses fine if they are given to poor people? If I buy one copy of a book and then print many copies of it myself and give them out is that fine? No none of that is fine, it is all theft.

That code you are writing with speech recognition, are you actually speaking every word and piece of punctuation yourself or are you basically telling the LLM how you want other people's code stitched together?

Everything you described can be done legally without using people's stuff without permission. Yes it won't be as easy and will be more expensive but it is possible.

Weather forecast models shouldn't have any problem with finding legal training data, neither should medical models, neither should financial models. Your statement that, "all the systems are based on a mixture of public domain and copyrighted information", has absolutely nothing to back it up. Also the medical industry has a requirement for privacy and laws specifically made to deal with medical privacy, at least where I am if anyone outside of who is actually treating me wants access to any of my medical records for any reason then they have to ask permission.

It has also been found through research that if you prepare the training data better and are more selective with it then it is possible to train a better or just as good but smaller models on less training data. The data can be prepared better by manual detailed labelling as one example that can be used for images.

For good speech recognition then why do you need to use data without permission? Either hire people to create training data or ask for volunteers and market it as helping disabled people. Ask people to spend a little time, maybe even just 5 minutes, reading out some text to be used as training data. This is something that people like you could do for yourself since if it is just your hands that don't work well then you can still read and speak. Yes it would be more work than just stealing every bit of audio with a transcript that you could find online but it is the morally and ethically better way of doing it. There would also be the requirement to use the data for a specific purpose and nothing else, like if it is collected for speech recognition then it can't be used for text to speech without asking for permission again. If it is code that is collected for a code speech recognition model then it is only used for that, not to train a code generator model.

There is no excuse for stealing, even if you believe it is being used for good. There are ways to get training data in a moral and ethical way, but these companies don't because that is more work and would cost more money.

Re: Delete them

It doesn't matter how significant the data is or is not, it is illegal to use it like that. Also if a lot of the data is used without permission and illegally then it should all be removed, so even if an individuals data is insignificant all of the illegally used data combined is definitely not insignificant. You talk like it should be an opt out system after the fact, why? Why shouldn't it be an opt in system like it morally should be?

Since you seem to have all the answers. How do you remove training data after the model has been trained? It's not a simple task and likely isn't even possible due to how the models are trained and would involve removing the data from the training data and retraining the model which is not feasible to do every time someone wants their data removed considering the cost and time required to train the model.

Plagiarism is deliberate copying or stealing of ideas, it has nothing to do with how unique a concept is. If someone's IP is not unique that doesn't mean they plagiarised. There are many examples of the same technology or ideas being developed independently at the same time.

Abstract, theoretical computing qualifications are turning teens off

"The GCSE contained out-of-date content about networks and internet protocols that could be removed from the specification"

What out of date content would this be? The Internet and other networks still work very similarly to how they did when they were invented, most of the content should still be relevant especially content taught at GCSE level.

Or is this out of date just because most people don't need to know this? Instead they just use a library and it handles everything?

Re: taught to get the best out of tools like ChatGPT so they can succeed in life and modern careers

They might well have good learning material on how AI works but that isn't the point. They want a separate course that is just about how to use computers, not computer science and there is a very good chance that it wouldn't go into how AI works just how to use things like chatGPT. So at best it would be surface level knowledge on how to use particular models.

Qualcomm's Windows on Arm push would be great – if only it ran all your software

Re: Value proposition

Microsoft themselves make a number of ARM powered devices, so they are invested in it.

UK orders Chinese biz to sell majority stake in Scottish chipmaker

Re: Why is the order from the Chancellor of the Duchy of Lancaster?

Did you read the article?

"The Chancellor of the Duchy of Lancaster is the highest-ranking minister in the British Cabinet Office after the Prime Minister."

Hide the keyboard – it's the only way to keep this software running

Re: Extremely bad design

"hence the shield instead if, say, locking it in a cupboard"

If the keyboard wasn't needed for day to day operation then there is a much simpler and more robust solution. Just unplug the keyboard.

Re: Wat?

He through his lunchbox on the desk and hit the keyboard, it's not like the table just got bumped. For it to happen multiple days in a row the guy must have done it multiple times, hence just throwing things around and hitting random buttons.

Combustion engines grind Linus Torvalds' gears

'EV motors are “simpler,” and manufacturers don’t need a “decade of experience to make a good electric motor,”'

Spoken like someone who doesn't know what they are talking about. Electric motors are far from simple and don't exist in a vacuum either, they need a lot that goes along with them.

Sure you could make a basic DC motor and run it with simple drive electronics but you wouldn't get good results. Motors are very complex when you start trying to optimise for power and efficiency, especially when you also try to make them light, compact and reliable. Then add in the battery and it's protection and monitoring and charging systems, where they are trying to pack as much capacity in as they can and still keep it small and light, whilst still being able to provide high current. Then add in the very complicated area of motor drivers and regenerative braking, designing a motor driver to handle that much power and do it efficiently and handle regenerative braking is far from simple.

Then add in all the other parts of a car, air bags, crumple zones, brakes and all the electronics and that is far from simple too and it needs to be very reliable and fail safely and pass all kinds of regulations. Why do you think most EV manufacturers are ICE car manufacturers?

I have no doubt that he would have a lot to say if someone called core parts of the linux kernel simple and said that they don't require much experience to make. I would have thought that with him being such an experienced developer and managing a large scale open source project that he would realise that things aren't that simple. It is a common annoyance with engineers and software developers when other people who don't know what they are talking about call something simple or easy when it really isn't.

Re: It's not the electric motor

For some reason I have seen that a lot of people seem to think that you can use a Tesla or other EVs 12 V battery to jump start an ICE and have heard of people actually doing it. Whilst it may work the batteries aren't designed for it and it will just degrade them and the manufacturers don't recommend it either

Re: It's not the electric motor

It is probably a safety thing. By having a small 12 V battery they can completely disconnect the main battery (from everything except the BMS) when not needed. It also means they don't need to deal with the inefficiencies of a multiple hundred volt to 12 V converter when the car is just sat there since the quiescent current would just end up draining the main battery.

Re: Linus knows way too much about software

But knows very little about car design. You would think someone with his experience working on a large scale project with lots of different interoperating systems he would realise that the engine/motor of a car isn't the only thing needed and that an electric motor, battery and motor driver is actually quite complicated. I bet he would have a lot to say if someone refered to certain parts of the linux kernel as simple and could be done without much experience.

Re: the rumble of the V8

It was a big problem with early EVs, they were just too quiet and they were expensive meaning it was only a certain kind of rich person who had them and they weren't usually the most alert when driving and didn't go very slowly through towns.

Re: Makes perfect sense

Whilst ICE are not very efficient, EVs aren't all that efficient either once you take into account all the steps and losses, from generation to transmission to the EV charger to the batteries and finally to the motors. Most sources I saw claimed an EV itself was around 70-80 % efficient which when stacked will all the other inefficiencies. Obviously fuel engines have similar transmission and extraction losses too though.

Due to the nature of EVs there is only so much they can do to make them efficient since they need to be light and compact.

Then there is the issue of batteries, particularly lithium ones, they just aren't good for the environment due to how they are produced. Batteries are what hold most electric vehicles back, especially aircraft, they are just nowhere near as power dense as fuels. Hydrogen is likely a better choice long term. Faster to refill and doesn't require large amounts of lithium batteries, plus it is clean too, the only issue is storage.

Re: Hmmmm...

It was such a dumb comment about something he clearly knows very little about.

Your comment about regenerative braking misses something out. Nothing is 100% efficient and so if you accelerate with a full battery up to 60 mph and then regeneratively brake back to 0 you won't end up with a full battery. The only time what you mentioned would apply would be if you started fully charged on a hill going downwards. There are other ways of braking using the motor too, if I'm stead of regeneratively braking you instead short the motor windings together then the motor will brake too. Also all EVs require friction brakes anyway for emergency stops and incase the motor or motor driver fails and in case the regenerative braking isn't aggressive enough.

San Francisco billboards call out tech firms for not paying for open source

Re: The trouble with money

For the most part I agree with you, but I do still believe it could be used badly. Even just a project being taken in a direction that isn't beneficial for most people or in the original purpose of the project just because a company is giving the developer money. At what point does it just become writing code for the company?

It all depends on the integrity, stability and judgement of the developer. I could see it happen quite easily, a company threatens to pull funding that the developer needs to continue or sustain themselves if the developer doesn't do what the company wants.

I can also see it turning into a competition between companies. They each try to donate more than the others to try and get priority on their requests. If could be very good money for the developer but it may not be worth it.

Yes developers need to eat, so they should get jobs or charge for their code rather than just release it for free. They could also offer additional services like support or hosting. If their project is large enough then maybe they can sustain themselves on donations but that only happens with the largest projects and it shouldn't be expected for every project or piece of open source code.

Why do people release code as open source? It shouldn't be so that they can make money on it.

Relying on donations for your open source project to sustain yourself is not a great idea anyway. What if someone else releases something better or even just forks yours and improves it? What happens if a company releases closed source software that is much better? Realistically your project could become obsolete very quickly and then the donations will drastically decline.

Re: The trouble with money

Not even going as far as backdoors just companies expecting the developers to do what the company wants and using money like you described to try and force the developer to do what they want.

The open secret of open washing – why companies pretend to be open source

Re: OpenAI

OpenAI is particularly bad. Yes they started as being open but for a long time now they haven't been but won't change their company name, likely due to the public perception of it having "open" in it.

There are no protections for this sort of name though and that has meant a few companies have done similar, start off being at least partly open with "open" in the name and then go closed source.