This is Linux ... I know this!
For readers old enough to remember ;-)
276 publicly visible posts • joined 12 Jun 2009
"It is, of course, getting worse. The whole idea of agentic AI is pinned to the donkey by the assumption that your agents need your access rights to act on your behalf. There being no industry-wide best practices, no inherent management principles, or indeed inherent anything, this means giving AI agents your passwords – something that in a sane and godly world you would not do."
No industry-wide best practices? You mean, like OAUTH, which has been around for two decades?
My tale is from the early 1990s, before PowerPoint was invented to bring joy into our lives, and presentations were done using acetate sheets on an overhead projector (kids, go ask your grandparents).
Our department's very expensive photocopier was rendered unusable when someone fed regular acetate sheets through it instead of the special heat-resistant ones. The sheets simply melted onto the drum. The perpetrator had fled the scene, but he had left the originals of his slides in the machine, so the departmental administrator was able to identify him and deliver an appropriate bollocking.
About 20 years ago, the Open University ran an excellent course called "Learning from Information System Failures" as part of its postgraduate computing programme. It examined the different ways in which large IT projects often fail. The course material included several case studies, including the infamous Cambridge University CAPSA project and the Home Office's 1999 Passport Office fiasco. If the OU ever brings back this course, I predict that the Digital ID project will be a prime candidate to become a new case study.
It looks like Ray committed a file containing production database login credentials into the company's source code repository. That is shoddy practice on so many levels. If Ray had worked in my organisation, he'd be getting a very severe bollocking from our cybersecurity team *and* our DBA team.
The Home Office is already trying, and failing, to implement a digital ID system for millions of foreign permanent residents living in the UK. It's called the eVisa system, and it is riddled with flaws. The campaign group the3million has recently published a detailed report on the failings of the eVisa system (https://the3million.org.uk/digital-status-crisis) which should be required reading for anyone advocating for a nationwide digital ID system.
On the plus side, given the track record of large, complex government IT projects, this one is never likely to get off the ground.
All major relational database systems allow DBAs to control what each user account can do by granting different levels of privilege, all the way from superuser status down to "you can connect to the database but you can't see or alter any data". So why -- especially after the first incident -- didn't the DBAs severely restrict the reporting account to allow it only to execute SELECT? That's what we do in my organisation. If you have a valid need to view data, but not to alter it, we grant you SELECT privileges only. You can try to execute INSERT or UPDATE or DELETE, but the database will respond with "you don't have the privileges to do that".
It's called an eVisa, and the Home Office has been issuing them to foreign students, workers and non-British partners of British citizens since last year. Existing non-British residents with physical immigration documents, such as stamps in passports or biometric residence permits, are now required to exchange them for an eVisa.
Employers and landlords are legally required to verify the eVisa status of anyone they want to hire or rent to. It is also checked by airline staff before boarding a flight to the U.K.
The Open Rights Group published a report last year (https://www.openrightsgroup.org/press-releases/broken-e-visa-scheme-could-lead-to-digital-windrush-scandal/) highlighting serious problems with the Home Office's implementation of the system, and warned that it could lead to another Windrush scandal. Readers of The Register will find the ORG's report both fascinating and alarming, as an example of a major government IT project that has fatal flaws.
Wilma was extremely lucky that there were backups of the other web sites, but her first mistake was to begin by DELETING the old web site directory, when a more experienced person would have RENAMED the directory e.g. "mv website website.old", before installing the new web site alongside it. That way, if the new version didn't actually work in the production environment, the old web site could be restored in seconds with a couple of directory renames.
Still, the wise learn by their mistakes.
"We may live in a galaxy teeming with life, but very little multicellular or intelligent life."
As the Monty Python team once memorably expressed it:
So remember when you're feeling very small and insecure,
How amazingly unlikely is your birth,
And pray that there's intelligent life somewhere up in space,
'Cause there's bugger all down here on earth.
Bu those legacy systems also contain undocumented workflows and business logic due to (in the words of the article) 'encode[d] arcane policy'. You could be a COBOL grandmaster, and still not be able to figure out what the f**k the program is doing. Anyone who has ever been told to debug someone else's 10,000-line program will know what I'm talking about.
Considering the billions of pounds that have been wasted trying to integrate NHS medical records over the years so that Hospital A can view your records from Hospital B or GP Surgery C, it's ironic that at least one hospital in the U.S. appears to have managed it quite by accident.
Twenty years ago, the Open University offered an excellent six-month graduate-level course called "Learning from Information System Failures". The course materials included several case studies of large IT projects that had failed. One was the disastrous Cambridge University Capsa project of the late 1990s, which attempted to implement a single accounting system across all the colleges and departments of the university. If the OU ever revives the course, Birmingham City Council's Oracle ERP project will surely be a prime case study.
As a Ph.D. student at Liverpool University in the mid 1980s, I tried to use the X.25 JANET FTP protocol to transfer a set of data files from a VAX at the Royal Greenwich Observstory (of blessed and glorious memory) at Herstmonceux to the IBM mainframe at Liverpool. It turned out to be a non-trivial task, and even when the head of operations at Liverpool got involved, he was unable to get it to work, and advised me to ask my collaborators at the RGO to put the files on tape and send it by Royal Mail.
There are Babylonian clay tablets containing cuneiform text that are almost 4,000 years old and still readable. Some of them contain records of solar eclipses and other astronomical events that are still being used by modern astronomers to investigate how the Earth's rotation rate has changed over the millennia. Now THAT is durable data storage!
If I were cynical, I might wonder whether this is simply a case of AI being used to generate greater volumes of the kind of corporate BS that blights the lives of the people who do the real work. I think readers of El Reg will know what I mean: emails, reports and presentations from HR, senior management and consultants, full of buzzwords but oddly devoid of any actual meaning. On the plus side, if I can toss these into Copilot and ask it for a one-line summary in language that a 7-year-old can understand, I'll save myself actually having to read all of the tedious crap and save myself a lot more than 26 minutes each day.
Ah, you're talking about MyISAM, the bane of every MySQL DBA's life. Yes, legacy applications that rely on MyISAM are a world-class PITA. I used to work with a team that refused to switch to InnoDB because they liked to move entire schemas between instances using filesystem-level copying of the data files. You could (with care!) do that with MyISAM. The manual says that you can now do it with InnoDB using the "Transportable Tablespace" feature, but any user who comes to me and asks about that will get the Paddington Bear Hard Stare.
"MySQL broke this by supporting, but ignoring the syntax, and using table scans whenever tables were joined, causing many developers to think, not unreasonably, that JOINs were the problem."
I've read and re-read this several times, but I still don't understand what you're saying. MySQL has always used indexes to perform joins, if there are suitable indexes. Its query analyser may not always pick the best index for the job, but it does have the EXPLAIN command, and an experienced developer or DBA will always run a new JOIN query through EXPLAIN to find out what indexes the query analyser is planning to use. MySQL also has a handy extension called index hints, which allow you to say to MySQL "no, don't use that index, use this index instead", when EXPLAIN shows you that it's picking the wrong index itself.
Occasionally, the query analyser will determine that none of the available indexes will give better performance than a full table scan. That can happen if you have an index on a column that has only a couple of values, and the data are split 50:50 between them. That index is pretty useless, and a full table scan will in fact be faster. That's not MySQL's fault, of course, but trying to explain that to some developers is like trying to explain quantum mechanics to my cat. Pointless and annoying for both of us.
"It is a long time ago but I remember having to fix a website where a single user had two different ids… and the root cause was the lack of data integrity across tables that a FOREIGN KEY would have ensured."
I've seen this myself, back when Ruby-on-Rails was (briefly) the must-have framework that all the cool kids were using. I was told, by someone whose job title was "senior developer", that they had no need for foreign key constraints, because ActiveRecord (RoR's ORM layer) took care of data integrity. A couple of months later, the team leader asked me to look at the possibility of adding foreign keys. If you know MySQL, then you'll know that it won't allow you to add a foreign key if it would be violated by existing data. That was the case with several of the critical tables, so I had to tell the team leader that the data were already inconsistent and would need to be cleaned up first. Thankfully, that wasn't my problem.
And that's why, whenever I hear a developer say "MySQL has changed my data!", I roll my eyes.
Well, the developers *would* say that, wouldn't they. After all, every developer will tell you that their code is perfect, and if weird sh*t happens, it must be the database, right? :-)
Seriously, if there were a problem like the one you describe -- MySQL randomly changing data -- then support forums like Stack Overflow would be full of threads on this subject, and it would be front-page news here at The Register. Googling for reports of MySQL changing users' data unexpectedly, I found just one Stack Overflow thread (https://stackoverflow.com/questions/49594025/mysql-loss-of-data) where a user claimed that MySQL was changing their data, and the leading reply was as skeptical of the claim as I am.
Occam's Razor says that the simplest explanation is the most likely: the bug is in the application, not the database.
"PostgreSQL's backup facilities are to say the least primitive"
That may have been true ten years ago, but the pg_basebackup tool has been part of the standard PostgreSQL distribution for more than a decade now. It allows you to make hot backups of both local and remote PostgreSQL clusters, and re-building a working database cluster from the resulting backup fileset is so easy that even an intern could be trusted not to screw it up. The capability to backup a remote database cluster also makes pg_basebackup the perfect tool for setting up standby clusters quickly and easily. I'd hardly characterise those kinds of capabilities as "primitive".
"I am speaking of experience with a database holding 3 million entities (a song metadata database). It would always lose records and never knew the exact number of songs."
Is it possible, do you suppose, that your software developers weren't very good at their jobs, and the reason why the records went missing is because the application was defective?
I ask because I've worked with MySQL for 25 years, as an application developer and as a DBA, supporting databases with tables holding hundreds of millions of rows, and I've never seen the kind of behaviour that you describe.
NASA had plenty of rapid unplanned disassemblies in the early days. In the movie "The Right Stuff", those clips of rockets exploding just seconds after lift-off are real NASA archive footage. The courage of the Mercury astronauts, especially Alan Shepard, cannot be overstated. Those guys watched rockets explode, then went ahead and sat atop one anyway.
Around 30 years ago, I worked for a while at a large UK university. The HR department had a reputation for leaving people on the payroll after they had moved elsewhere, so in my last month, several memos were sent to HR to remind them to remove David Harper from the payroll. A few months later, I started getting mail forwarded to me from the university, but for a David Harper in a completely different department. I emailed him to let him know that I had his mail and would send it back at once. He wrote back to tell me that after I left, HR had stopped paying *him* too. The poor guy missed two months' salary before it was sorted out. Not funny for him at the time, obviously!
As reported back in September by the Open Rights Group and the3million (https://the3million.org.uk/sites/default/files/documents/Loss%20and%20Liability%20-%20Glitching%20immigration%20status%20as%20a%20feature%20of%20the%20British%20border%20after%20Brexit%20-%20Sep2024.pdf) and more recently in the Independent (https://www.independent.co.uk/news/uk/home-news/evisa-uk-immigration-status-help-b2678643.html).
The Home Office's euphemism for this is entanglement. It sounds almost quaint until you remember that it means that your personal data is being shown to a complete stranger, or a complete stranger's personal data is being used to deny you the right to return to your home in the UK.