Re: Wasted potential - there was clearly mismanagement but it wasn't all bad
> However, they were genuinely trying to get building blocks to quickly assemble apps and AI code to convert the adaptations to those blocks from English-language requirements to code. The issue was that they didn't also manage to incentivise the contractors to actually use any of the generated code, because being billed by the hour and given no particular need to use that code they preferred to generally throw it out and write things from scratch. This meant that the ability to work out how close they were and refine that conversion was very limited.
Assuming that is a reasonable description of their process, it raises so many questions, which includes:
They were getting the AI to generate an app, the paying contractors to fix the app (presumably to the original requirements) then planning to use the difference between the two as the metric?
Assuming that the contractors were going to hit the requirements bang on each time? Or were they feeding back in the list of the AI's failings plus (different code and list of its failings)? Instead of letting the LLM "figure it out for itself" and only iterating over the match to requirements. See below about what they may actually be training this LLM to do...
> they didn't also manage to incentivise the contractors to actually use any of the generated code
That sounds like one of the things that needed to have been figured out the day *before* engaging the first contractor! And thoroughly tested as a mechanism before making any attempt to include it as part of the LLM's training protocol. Let me guess, the first year they used in-house coders, who were salaried , informed about the purpose of their work (being part of the training protocol) and were all enthusiastic about "getting the LLM to work" as their goal, as opposed to just chucking out yet another tedious app. Then the contractors were hired and they didn't have the same goals...
No BOFH clauses like "the rate we pay is divided by the number of changed lines"? And this incentive going directly to the individual contractor, not via middlemen (i.e. a contracting firm bigger than a one-man band)?
And we're 100% sure it was not because the LLM was generating gibberish and it was actually genuinely easier to start from scratch rather than try to figure out what was actually going on? Because there is a sneaking suspicion that the LLM is effectively being trained to generate code that REQUIRES humans to correct it and it may be optimising its outputs to maximise that!
> This meant that the ability to work out how close they were and refine that conversion was very limited. They had some spectacular cost over-runs on certain projects, again from the contract side.
Running projects for paying clients whilst they still had contractors in the mix, instead of a functioning AI? (If that was even going to be possible...)
> To be charitable, it did look like they generally took the hit when they took on a project for a fixed price and contractors ending up costing too much
Nothing charitable there, they took on the clients without having a working LLM. if they were hoping to use those clients as part of the training, then that is simply the cost of training. Otherwise, it is the cost of not going back to the client and admitting that they had sold them a service that didn't exist. Isn't there a word for that, begins with 'f', rhymes with 'Lord' (as in "have mercy").
Looking back at that lot, maybe the contractors did have this company by the short and curlies after all.
Biting the hand that feeds IT
