The joys of third-party applications.
Some years ago, I ran the web servers for a members' organisation.
This organisation decided that it needed a CMS. I recommended one, with a few others as backups in case they didn't like that one. But a Shiny Salesman turned up, and sold them a bespoke solution.
One fine afternoon, almost all of the CMS disappeared. The server was still up, but the pages were absent.
I was called in to find out what had gone wrong. It was rather shocking. It turns out the PHP had[1] two ways to get hold of environment variables - with a bright red warning on the documentation page never to mix the two, as context leakage would surely ensue. And, of course, the developers of this CMS had done exactly that.
Now, a page editor had (accidentally) included a link to his admin-area stuff, rather than the customer-side view of that page. That should have been harmless - no-one without some sort of administrative privilege should have been able to get to the admin side, so that's safe, right? Nope. This leakage meant that a user could accidentally gain administrative privilege if an admin was logged in at the same time. Guess which user did gain said privilege? An aggressive web spider, that merrily followed all the "delete page" links it found...
I put a patch in place to prevent re-occurrence whilst the developers "urgently" fixed the problem, and restored the DB from a copy I had secretly stashed away. The patch was still there when the CMS was retired, and no backup strategy was ever formally implemented. The developers in question have now discarded their product and are now shipping the one I'd recommended in the first place...
So has that organisation learnt from this? Have they hell.
A few years later, they decided they should have a CMS. I didn't even hear about the discussions until the deal was done, so by the time I asked "what about the one you've already paid for?", it was all too late. Another Shiny Salesman had done the dirty, and taken a large sack of loot away. And so the day of the rollout came around. One of the important parts of the new site was a Branch Finder application, that allowed users to find their nearest club. It was a Google Maps thing, and the developers[2] were very proud of it, as were my customers. So when users started reporting that it was *incredibly* slow, or didn't work at all, there was pandemonium. The developers, of course, blamed the server platform; I'd obviously commissioned something far too slow, and a new server was required. So I showed them the idle time graph to demonstrate just how little this server was doing; it was most certainly not a server problem. Then they decided that this was an inherent problem with the way maps work, and nothing could be done.
All this, of course, bent the needle on my Bullshitometer. A little interaction with the users showed that it was the ones with older PCs that were having most problems - it looked like a client-side problem. So I took a look at the application. The despair has not yet left me.
The application worked by sending the entire dataset of clubs - including data that would never make it onto the map, and probably breaches the DPA - to the client, where it is filtered for proximity to the user, and then displayed on the map. And the data is sent from the server in XML. Which is parsed on the client. In Javascript.
My initial replacement simply exchanged JSON for XML, and that went like the proverbial excrement from agricultural implement by comparison. That got rid of the "inherent problem" bullshit, and the developers resolved to have another look. By the time I left the project, they'd done nothing more than my quick hack...
Vic.
[1] I believe this is no longer the case in the current version of PHP. But I still wouldn't put any money on it.
[2] A differnet bunch from the first story. No better at their jobs, though.
Biting the hand that feeds IT
