Microsoft would read: 'It could look at the damage done and say: "We can do better."'
as
"We can do better, we have not yet damaged the world enough. Let us redesign and add even more bugs to the product and gaze upon the carnage."
Excel Hell: It's not just blame for pandemic pandemonium being spread between the sheets
The howls of disbelieving, horrified laughter caused by the news of the latest pandemic data cock-up yesterday were well deserved. 16,000 cases lost – purportedly in a blunder involving CSV data, row limits, and an out-of-date Excel file format? In a multibillion-pound, "world-beating" contact-tracing system? Unnoticed for a …
-
Tuesday 6th October 2020 13:33 GMT kfpm
Relax...
Ok, so you don't like Excel. You've made that perfectly clear. Like most IT tools, languages, there are quirks, nuances, whatever you like to call them. Excel serves a purpose. I hate to use the phrase 'in skilled hands', but that's the important thing. If you know what you're doing, Excel is a very helpful tool. It's not a corporate RDBMS replacement, but, unfortunately, that seems to get overlooked by those who want quick and dirty, and cheap, solutions to their problems. And that's why it's out there, and will stay out there.
-
-
Tuesday 6th October 2020 14:27 GMT Yet Another Anonymous coward
Re: Relax...
> almost all the actual uses of Excel in practice are as a simple spreadsheet
Almost all actual uses of Excel are as an ETL (extract-transform-load) tool.
ie to download information from one app, reformat/reorganise it and send it to some other app - and it's extraordinarily bad at this - in ways that most users don't know.
Hence all the genes misclassified as dates, dropped initial zeros in phone numbers, bad handling of embedded commas etc
-
-
-
Wednesday 7th October 2020 15:22 GMT Wayland
Re: Relax...
I used a tool called Monarch. This took printed (text saved to a file) reports from old mainframe type systems and turned them back into tables for import into a database. You had to create a template over the report to tell it which bits where page headings and table headings and actual tabular data. 20 years ago we use it to put purchase orders from a 1980's minicomputer onto a web portal.
-
-
Tuesday 6th October 2020 18:41 GMT Dr Paul Taylor
If your only tool is a hammer
People actually use Excel for things like modeling, tabulation, accounting and so forth.
then every problem looks like a nail.
If you have tabular data -- where the rows and columns have clearly established meanings as numbers or strings -- then ok use a spreadsheat.
Otherwise use an appropriate tool.
-
Wednesday 7th October 2020 12:57 GMT Norman Nescio
Re: If your only tool is a hammer
If you have tabular data -- where the rows and columns have clearly established meanings as numbers or strings -- then ok use a spreadsheat.
Nope. Excel cannot even cope with tabular data where you want to use logical operations across columns. Give me a count of instances where field 1 has value w, field 2 has value x or y and field 3 has value z, and field 4 is not any of a, b, or c. Loop through all unique values in field 1. It is possible to do with some pretty hairy expressions, but if you are trying to produce a summary tab that works on data in other tabs, or, heaven forfend, external files, you are in for a world of pain.
-
Wednesday 7th October 2020 14:10 GMT c1ue
Re: If your only tool is a hammer
Wrong.
Excel is powerful because Ajax allows it to reach out and access/do all manner of things.
I've created an Excel spreadsheet that would update, from a web page, the prices of my mother's 300 stocks, because her online brokerage account was so crap to understand.
How else would you do something like that? Particularly 20 years ago...
Yes, you can hire a programmer to slap together something custom - but what a waste of time and money.
Sure, it is easier to work with data these days particularly if there is an XML or similar type feed - but businesses existed before this and the Excel pages are an IT artifact just like the XP laptops fronting 15 year old Xray machines.
-
-
-
-
Tuesday 6th October 2020 22:15 GMT Marshalltown
Re: Relax...
I have to disagree. It is more or less OK as a data entry system, if you're not very fussy. But once you start to do anything practical things can go pear shaped in a blink. I was encouraged long ago in the dawn of Windows to use Excel and "shut up." So, simple statistics - came with an add-on set of statistical tools. What could be difficult about calculating the standard deviation of a set of artifact lengths? It is after all simply the square root of the variance, which is mean of the summed squared deviations. There aren't many. I could do it easily by hand, or right there in the spreadsheet. But there's a tool. So - what's this? A negative variance? That can't be right. How can that happen? There's no place for the number "i" in a simple sum of measures. I ran the calculations by hand and there the error was again! Square all deviations, they look right, sum them, still look right, divide to get the mean (the variance) and- where did that negative sign come from? I switched to R right there and then. Later I was told that the problem was probably due to some underflow or overflow condition. All I know is that impossible numbers are no use.
-
Wednesday 7th October 2020 12:46 GMT Norman Nescio
Re: Relax...
While I agree with most of what you say, it's worth noting that almost all the actual uses of Excel in practice are as a simple spreadsheet. Which is undeniably useful, and something Excel is pretty good at. That's why it's ubiquitous.
A good, simple spreadsheet would be useful.
Excel is not a good, simple spreadsheet. It has needless 'helpful' complexity, such as interpreting numbers as dates and applying other formatting rules in difficult to reverse ways. Inserting new, and cutting and pasting rows and columns can have unexpected and unintended effects that are not immediately apparent. Referencing data in external flat files and other spreadsheets is incredibly painful, so you are forced to take unmaintained copies of masters and import the data.
It is ubiquitous because it comes 'free' in Microsoft Office, not because it is any good, but it is just functional enough to prevent people looking for better choices.
-
Thursday 8th October 2020 14:29 GMT EnviableOne
Re: Relax...
It is ubiquitous because it comes 'free' in Microsoft Office, not because it is any good, but it is just functional enough to prevent people looking for better choices.
This is the crux of the issue, microsoft does make better choices Access, SQL, Project, Visio, PowerBI, Dynamics, etc. but they cost extra and not everyone has them, and Excel just about does the job, and is nigh on everyone has it.
Like Word is no longer just a Word Processor, Excel is no longer just a Spreadsheet, it has small parts of all the specialist tools shoehorned in, to make it just usuable enough that finance make you use it instead of paying for the right tool for the job.
-
-
-
Tuesday 6th October 2020 14:18 GMT Dan 55
Re: Relax...
This is a pandemic. It is serious stuff. The non programmer should ask a programmer when they need data crunching done, not do the car-racing equivalent of saying, "yeah, I can do that too" and driving their Ford Fiesta out onto the Formula 1 racetrack and start racing with them.
The programmer may say "you can use Excel for a while but I need the budget to do a replacement now because one day this will fail and you probably won't even know", that would be better than the non-programmers not knowing, congratulating themselves on a job well done getting the Ford Fiesta out onto the racetrack, and driving the car off the curve and into the audience.
I used a gratuitous car analogy, sorry.
-
-
Tuesday 6th October 2020 16:25 GMT Loyal Commenter
Re: Relax...
Yes, cos the intern at the NHS in charge of getting data emailed from 17 incompatible NHS trusts into one report is "empowered" (sorry) to call up Crapita and demand a custom applications
No, they are absolutely not, but the chain of manglement above them is absolutely responsible. Possibly just their immediate manager telling them to "get the job done I don't care how," or maybe right the way to the top of the organisation, depending on how ham-strung the intermediaries are by political edicts.
Someone is to blame, however, and it isn't the poor intern.
-
-
-
Wednesday 7th October 2020 10:08 GMT Loyal Commenter
Re: Someone is to blame
Indeed. With the utter incompetence of the Current Lot, it's easy to forget that the Previous Lot (who happened to include some of the Current Lot) were also pretty bad. And PHE was put in place by them, no doubt with some stooge at the top who is there not because of their competence, but because of their political connections.
And we think Russia is corrupt...
-
Thursday 8th October 2020 14:41 GMT EnviableOne
Re: Someone is to blame
She is, despite Duncan Selbie still being notianlly head of PHE, PHE have been rolled into the new National Institute for Health Protection (NIHP) which She is Interim head of....
But still the blame has to land at the feet of Andrew Lansley and the Health and Social Care Act, that ripped the Public health boards apart and gave the responsibilities to local councils, without the funding, and deposited the central parts in PHE, so it was basically a controll centre for something that no-longer existed.
Maintaining a Public Health infrastructure is expensive, but when an epi or pan demic comes along it repays that in spades, but if you dont have one, it starts to look like a good line item to cut in a budget
-
-
Wednesday 7th October 2020 14:46 GMT Kane
Re: Someone is to blame
"Someone is to blame... Dido Harding"
Twat.
Sorry, not you; I have a slightly nervous reaction to the words Dido Harding, in that I involuntarily type "Twat" whenever I see it on screen.
Twat.
See? Did it again! I really should write that custom dictionary that converts her name onscreen into Dildo Hardening or something to prevent this from happening in the future.
Twat.
Dammit, that didn't work either...
-
-
-
-
Wednesday 7th October 2020 07:59 GMT Persona
Re: Relax...
The programmer may say you can use Excel for a while but I need the budget to do a replacement now because one day this will fail and you probably won't even know
The programmer then went on to code the Post Office accounting system and some users were convicted of fraud when the money that supposedly had been collected was not there.
There are bugs everywhere. Government IT solutions have a habit of taking years, getting cancelled before they deliver anything and when they do deliver the results are still wrong.
-
Wednesday 7th October 2020 23:11 GMT A.P. Veening
Re: Relax...
Government IT solutions have a habit of taking years, getting cancelled before they deliver anything and when they do deliver the results are still wrong.
For government IT solutions the available options are working, on time and within budget, just pick any zero out of three.
-
Wednesday 7th October 2020 08:32 GMT T. F. M. Reader
Re: Relax...
The non programmer should ask a programmer when they need data crunching done
I imagine a reality satisfying the following set of requirements and specifications.
Matt Hancock needs to make presentations of "stuff" to various audiences - PM, Cabinet, MPs, press, public at large, you name it. This is the primary requirement, any data analysis needed to understand and deal with the virus spread is by definition secondary.
To satisfy the primary requirement a flunky, probably with a title like "political advisor", decides what slices of data should be presented. This varies between presentations, probably many times a day. There may be more than one flunky - a committee - doesn't matter.
Another flunky needs to create graphs and bar charts and pie charts on demand, for each presentation, under pressure, because the first flunky tends to change the decisions with very little time to spare. If there is a committee there will be even less time. There is absolutely no way a programmer can be engaged for this. The only tool the flunky knows how to use is Excel. This means that whatever methods of collecting, storing, and processing data are used upstream the data must be available in Excel so that the geeky flunky will be able to produce new charts many times a day.
Upstream, there is a clear understanding what the primary requirement is, so Excel is chosen as a major, likely the primary, possibly the only way to collect and store data. Someone figures out that most secondary data processing tasks are not terribly complicated and can be done in Excel. No one knows that Excel's numerics and statistics are "quirky", and if anyone does they are told it is not an important enough consideration. "Best practices", as in, "everybody uses Excel", are probably mentioned.
Cynical? Moi?
-
Wednesday 7th October 2020 09:30 GMT Flywheel
Re: Relax...
Matt Hancock needs to make presentations of "stuff" to various audiences - PM, Cabinet, MPs, press, public at large, you name it
We have the British Computer Society (BCS) with a huge diversity of experience and knowledge among its members both in the UK and worldwide. We need Hancock to appear before that committee and see if his ideas hold up.
-
-
Wednesday 7th October 2020 09:26 GMT Flywheel
Re: Relax...
It's the B-word: Budget. If Baroness Harding can come out with nonsense like "We weren't expecting a [global] pandemic to create so many cases of illness and death" then she and Hancock certainly won't have allocated a budget for software and/or development. I can imagine Hancock telling his data minions to "upgrade to a later version of Excel with more rows" when the original version ran out of capacity.
-
-
Tuesday 6th October 2020 23:40 GMT Anonymous Coward
Re: Relax...
The sad thing is there are quick and dirty solutions out there...a simple bash script could have done what this fuckwit tried to build in Excel.
The fact of the matter is, the chap that implemented this solution was technically incompetent on every level.
I suspect the vast majority of us here could parse and process a CSV in any number of ways and it would take us a while to find someone that leapt straight at Excel.
Excel may have it's purpose...but as a batch processor for CSVs it is so ill suited that literally every techie everywhere hearing that it was used for this felt dirty.
I have no idea what they were parsing into, but I'm reasonably confident whatever it was has a command line tool available that you can pipe to.
cat covid_crap.csv | while read line
do
read -d, column_1 column_2 < <(echo $line)
# DO SOMETHING WITH THE DATA HERE, PIPE IT TO A SCRIPT IF YOU LIKE, IDGAF
done
Admittedly this isn't the most efficient way to parse a CSV, but it'll snarl through hundreds of thousands of rows in no time flat.
There's absolutely no need to have a fully featured GUI application in the middle. Zero.
Even if you had to write a Python or PHP script with a database connection to handle the data once it's parsed, it would take less time than building an Excel VBA macro.
What the Excel solution reeks of to me is the handy work of some academic intern using an example snippet from a text book somewhere. Nobody with actual real world hands on experience handling files larger than 1mb would ever look to Excel as a solution. Only someone unaware of it's limitations and a lack of knowledge outside of a Microsoft ecosystem. 65k rows in a CSV isn't even that big, I've kicked around CSVs with hundreds of millions of rows before with relatively little trouble...it never once crossed my mind to convert those CSVs into something else before I parsed them...I just parsed the CSVs.
The only situation I can see that Excel might be necessary is if there is some wanker .NET developer somewhere in the chain on the backend system hard coded the import to only use Excel spreadsheets for "compatibility" because he has never used CSV files before and had Excel installed on his laptop which enabled him to test with small datasets.
Apologies to all the .NET devs out there, there probably is a good one somewhere...but come on guys, you're the only developers that see throwing more RAM at a SQL server as being a solution. It's never your crap nested UNIONS and JOINS that are the problem. It's also never the complicated SPROCS performing logic that the client side application should be doing.
Also, if I had a quid for every time a .NET developer said "well it's NGINX that is displaying the error, the problem must be there, look it says BAD GATEWAY", I'd be a very rich man.
-
Wednesday 7th October 2020 09:39 GMT Christopher Reeve's Horse
Re: Relax...
Technically incompetent at every level?
Yes, your argument is very good, but typically the user of excel has an entirely different area of technical competence. You clearly have a technical competence for Bash scripts, and that's great, but the problem is the overlap of responsibilities and the management of resources, and probably not the technical competence of the original user.
-
Wednesday 7th October 2020 09:53 GMT David Nash
Re: Relax...
"someone unaware of it's limitations and a lack of knowledge outside of a Microsoft ecosystem."
I think you've hit the nail on the head there.
Someone who had to come up with a solution didn't have the knowledge or experience to know that there were better solutions available. "Excel can hold thousands of rows of data and it's on everyone's desktop, what's not to like?"
Non-techies probably wouldn't even consider that there may be a limit on the number of rows.
-
Wednesday 7th October 2020 10:12 GMT tfewster
Re: Relax...
@AC, your script fails if there's a comma in a data field, even if it's within quotes (the CSV standard).
Better to use something that understands the CSV standard, like, um...Excel? In my experience, it tells you if the CSV data hasn't completely loaded ( > 1M rows, so that might only be in later versions).
I believe PowerShell has an Import-CSV function. I expect there are Linux utilities that can do that too. I'd probably find them useful, as one of my duties involves producing a huge, meaningless
reportlisting which I then parse in a bash shell to pull out useful data.Excel is remarkably good at making up for systems that can't produce decent reports, though it has pitfalls. The real WTF is why a 65,000 row "report" is needed - a human won't read it. If it's using CSV to transfer data from a collection system to a proper analysis tool, that's fine. If Excel is your (temporary) analysis tool, just be aware of the limitations.
Downvote away if you have a ready-to-roll analysis tool for, say, < £500/seat ;-)
-
Wednesday 7th October 2020 13:06 GMT Norman Nescio
Re: Relax...
The sad thing is there are quick and dirty solutions out there...a simple bash script could have done what this fuckwit tried to build in Excel.
Speaking from experience here as a low paid administrator (I have other experience), the locked down laptop does not have bash available, and it is not available in the corporate-approved portfolio of applications available via IT. Neither is, for example, SQLite.
On many occasions I have been reduced to bringing in my personal laptop to get stuff done that simply wasn't possible using the corporate IT set-up available to end users. Can't do that with health data as GDPR and basic data privacy rules mean I can't move the data out of the 'secure' corporate environment - which is correct, and I would not break those rules. It makes life challenging.
-
Thursday 8th October 2020 13:54 GMT uccsoundman
Re: Relax...
You and I (and many others here) are the exception, but being a hiring manager for software development I am constantly astonished at how many CS graduates have never seen a command line interface. And in my company, until just a year ago, even MENTIONING Linux or Unix could get you fired; we're a windows house. So often it's just better to do what you are told and get the wrong answer than to risk your job by doing it right.
-
Sunday 11th October 2020 09:56 GMT Anonymous Coward
Re: Relax...
GDPR and privacy controls only apply if the data hasn't been anonymised effectively.
Assuming the track and trace folks only get personal information when they narrow down to a venue etc, there should be nothing in the data that is personally identifiable.
BYOD is no better than installing unsanctioned software on the corporate network. Either way, you're circumventing policy.
-
-
-
Wednesday 7th October 2020 07:54 GMT FlamingDeath
Re: Relax...
If you think their excel is a spawn from hell directly out of the anus of bill gates Arse™️ (Version 3.2, release 8)
Then consider the amount of electricity wasted on this fucking awful operating system while it attempts to complete an “update”. How many power stations are needed worldwide to accommodate this travesty ?
Oh, and Dr. Bill want to give as all vaccines, you couldn’t make this shit up
-
Wednesday 7th October 2020 12:23 GMT Cederic
Re: Relax...
Because the fault isn't with Microsoft.
The fault isn't with the Government, or the people managing that Department.
The fault isn't with the users that are using Excel.
The fault is one of IT strategy and governance. You know that users will use Excel, you know that they're going to use its competitors if you try and stop them, you know that they're going to do so because it enables them to do their job quicker and often better, you know that they have little choice because any time they ask IT to implement a solution it costs too much, takes too long and doesn't do the right damn thing anyway.
So where are the policies and procedures to control its use, where are the data standards and validation to prevent issues like this, where's the training to assure people don't misuse it, why aren't other tools provided that let people do data merges more reliably?
I hate the prevalence and misuse of Excel at organisations but there's only one part of the company able to do something about it.
-
-
-
-
Wednesday 7th October 2020 21:38 GMT AndyD 8-)₹
Thirty years ago...
... let me say that again - thirty years ago I wrote a report for a major bank, outlining the risks and drawbacks of using spreadsheets for almost anything!
This was a bank that had very rigorous very very thorough, very s l o w, but almost completely foolproof in-house software development procedures. My reportt was received 'politely'.
A few short months later they made swingeing redundancies across the IT department and brought in newer, faster fashionable techniques.
Hey Ho
-
Tuesday 6th October 2020 13:39 GMT Dave314159ggggdffsdds
This is just a teenage rant from 2000. I'm surprised the author doesn't refer to micro$oft.
Obviously most of it is hyperbolic verbiage, but the substantive parts are simply wrong. Excel is like mole grips: there are very few situations it's the really correct tool to use, but in an awful lot of others it's good enough to get the job done when you don't have the perfect tool.
Blaming the tool for someone's idiocy is silly; there are endless examples of idiots managing to idiot even with the right tools.
-
Tuesday 6th October 2020 14:27 GMT vtcodger
Downvoted because in my experience Excel adds a variety of bizarre idiosyncracies on top of the numerous normal hazards of spreadsheet use. I'm talking about things that didn't/don't happen in better behaved spreadsheets. My favorite -- the teacher who tried to copy and paste a column of zip codes. Excel inserted the first then copied down as expected. But it quietly incremented each code after the first 90210, 90211, 90212 ...
-
-
-
Wednesday 7th October 2020 12:26 GMT Cederic
If only you could hit ctrl-z and undo the clumsy edit.
Software design is a delicate balance between ease of use for inexperienced users, accessibility and enabling power users. Excel's autofill capability is not broken behaviour, even if you disagree with where they've tried to find that balance.
-
-
-
-
Tuesday 6th October 2020 14:31 GMT big_D
Having faced many abortions of "systems", "developed" in Excel, I can only agree with Rupert.
Using Excel as a proper spreadsheet to analyse some figures, fine.
But I've experienced:
* Timesheet system for a whole company automated with Excel 4 macros (not VBA, Excel Macros).
* Forecasting system so complex in 1-2-3 that Lotus threw up their hands in disgust and told my project manager to use a real language!
* Production completely bypassing an ERP system, doing planning and production in Excel and booking the end result back into ERP.
* Complex "databases" automated in Excel
And many more sins.
-
-
Tuesday 6th October 2020 22:46 GMT Marshalltown
Simple tables
In Word and some other current wp's you can type the values in as comma or tab separated columns, select the block and tell the word processor to convert it to a table. You can then the format the table in the wp. Or, you can simply copy a csv or tsv file, paste it into the wp and again convert to table. Vastly better than creating tables using the GUI and less aggravating than dealing with some of the weird incompatibilities encountered occasionally even between MS products.
-
-
Wednesday 7th October 2020 00:33 GMT Anonymous Coward
Yep. Where I am we use a "spreadsheet" to record hardware in racks at the data centre (not a list but cells made to look like the racks!), then when and SSL cert was forgotten and expired someone started up a new spreadsheet to track SSL certs (which is hardly ever checked or updated).
The other one is a particular gripe of mine - our incident and change management system has a thousand different specific data fields, yet most are irrelevant or unused in favour of copy+pasting some plain text templates into about 5 separate text areas and painstakingly filling in the blanks! For all change requests it's necessary to fill in a 'risk scorecard' which is... You guessed it! A f**king Excel spreadsheet with macros where you answer some multiple choice questions about the scale and impact of the change, press 'calculate' and then copy+paste the risk scorecard' back into a text box in the change request.
We're paying vast sums for a big name CoTS product and then using it like it's notepad.exe plus Excel to do what should be a built in form.
It's madness but everywhere seems to do similar horrible things :-(
-
Thursday 8th October 2020 20:23 GMT TRT
I sat in on a session today. Updating 785 network switches across 13 sites with 53 buildings.
The project was being managed using Excel. The consultant must have spent 60+ hours constructing an intricate balance of pivot charts and look up fields. At £400 am hour no doubt. I watched him live on a call with the stakeholders add in all the unidentified risks they'd missed. The whole thing collapsed as soon as he added a single line. And we pay for it.
-
-
-
-
-
Tuesday 6th October 2020 21:22 GMT TRT
Re: Ye Olde English Proverb
Excel is to tools as a Swiss Army Knife is to. Well. Tools. Handy, everyone has one, knows how to open it. And then proceeds to use it to cut down a tree, make planed timber, turn it into a flat pack and build a cabinet. But that must be the right tool as it even had a thing for getting stones out of hooves.
-
Tuesday 6th October 2020 22:36 GMT the Jim bloke
Re: Ye Olde English Proverb
I have used a swiss army knife to cut down a tree.
There was alcohol involved.
This also led to setting the stump on fire, and driving work vehicles across the smouldering stump. I cant remember just why that particular tree merited such treatment, but it must have been something heinous.
Generally I only used swiss army knives - and leatherman tools now, to remove branchs or shorten small lengths of timber, but that tree had it coming.
-
Wednesday 7th October 2020 07:21 GMT MattWPBS
Re: Ye Olde English Proverb
Swiss Army Knife is the analogy I always use.
You can trim branches with the little saw blade, you can slice up food with the knife blades. If you end up trying to run a lumber business with Swiss Army Knife though, you want to take a step back and think about what you're doing.
-
-
-
Tuesday 6th October 2020 23:08 GMT Anonymous Coward
Re: Ye Olde English Proverb
Re-arranging that logic, a bad workman is a tool, which means his managers can't blame him. Therefore those responsible for this mess are the managers. Unless his managers are also tools, in which case those above can't blame the managers...and so on.
It would appear that logically, an organisation full of tools is blameless...we have to look at how they got there...
Next GE everyone needs a long, hard think.
-
Tuesday 6th October 2020 16:03 GMT Dave K
The problem is that MS have added things to Excel to make it more versatile and have enabled these abuses of Excel. It does have its place and in certain situations, it is the correct tool.
I consider Excel to be a bit like a pair of mole grips with a multimeter, wrench, soldering iron etc. gaffer-taped to the handles. The functionality is sort of there, but that doesn't mean you should use those mole grips for soldering a circuit board together...
-
Tuesday 6th October 2020 16:38 GMT Loyal Commenter
... there are very few situations it's the really correct tool to use, but in an awful lot of others it's good enough to get the job done when you don't have the perfect tool.
The problem is that whilst being "good enough", it fails in often untraceable and unpredictable ways. So what you think might be "good enough"suddenly stops working or produces errors you don't even know about.
Like losing the results of thousands of positive coronavirus tests.
Unless you are an "expert user" of Excel, you may well not even know about the limitations it has that would produce these problems, so you, quite reasonably, might think it is a suitable tool for the job when it is not. This is the real evil of Excel.
That and lack of testability, reliability, traceability, and security in a half-arsed incomplete programming model. It allows people to think they are being clever getting "calculations" done without even realising that they are effectively doing a programming job, but without the benefit of knowing how to program, and the caveats that would bring, such as the need to define behaviour, boundary conditions, inputs, outputs, etc. and verify and test the behaviour to ensure it is correct. Actually, that - that is why Excel is Evil.
Oh, and people's thinking that it can be used as a simple ETL tool, with the assumption that if you export something to Excel and re-import it again into $software, opening the file before re-importing it won't suddenly decide to convert half your values into random numbers, or dates, or remove leading zeroes, drop precision from numbers, choke on dates before 1900 (whilst thinking that there was a 29th February in that year) and all other sorts of shenanigans. THAT is why Excel is EVIL.
I mean, even for its original purpose, of simple accounting spreadsheets, it has the complete lack of structure and control that you would get in any actual accounting software. It might be okay for reporting, with various caveats, but invariably, whenever I have come across it in the 25+ years I have been working with computers, there has been a better or more appropriate tool for the job. In summary, Excel is EVIL
-
Tuesday 6th October 2020 20:47 GMT Krassi
Excel is used all the time because the specialised tools don't work ! It is harsh to blame excel for being crap when it is being used as sticking plaster to patch up much more expensive and supposedly advanced database systems that can't do simple (*) tasks like passing data from one place to another.
(*) far from simple of course, but the objective is simple and easy to ask for.
-
-
Tuesday 6th October 2020 13:42 GMT WolfFan
I get it
It’s Microsoft’s fault. Hmm. I remember when a small, little-known, company named, ah, ‘Lotus’ brought out a far superior product named Jazz. I remember it crashing and burning, for several reasons: it was Mac-only in 1985, it cost $600 (in 1985! Jesus bloody Christ, what were they smoking at Lotus?), it was aggressively and yet uselessly copy protected (at the time the standard Mac floppy was 400 kB, but the Jazz disk 1 was 405 kB, making it impossible to copy... unless you had a copy of CopyIIMac or similar, which copied it in minutes) and, most crucially, it sold under 20,000 copies (see ‘Mac-only’, ‘$600’, and ‘outrageously copy-protected’) while the then brand-new Excel v1 sold over 200,000 copies because while it was Mac-only (until Windows 1 showed up, that is) it cost less than half what Jazz did and was not nearly as user-hostile. Excel also did much less than Jazz, but went on to become _the_ killer app on Macs, Win 1-3, Win 95, Win NT... Lotus canceled a follow-up, ‘Modern Jazz’, which would have been much better and possibly run on Win 1-3. Lotus also didn’t bring out a Mac version of 1-2-3 until 1992, and brought out a Windows version of 1-2-3 far too late. Borland Quatro crashed, burned, and went to Canada, becoming part of Word Perfect Office from Corel... and notably stayed away from Macs after Corel killed the Mac version of Word Perfect. Excel was left as last man standing, with no real competition, and not due to anything Microsoft did. Lotus did try again with Symphony, but by that time it was too little, too late.
People used Excel because there was no real competition and because it was what they had. If Lotus, or Borland, or _someone_ had put something worthwhile out as competition to Excel, then perhaps things would have been different.
-
Tuesday 6th October 2020 20:49 GMT Anonymous Coward
Re: I get it
Only one problem with you story. Thats not what actually happen. I know. Because I was there.
From the mid 80's to the mid 90's the Mac market was an afterthought in the business software market. It was 95% MS/DOS only. Then starting to make the transition to Windows with Win3.1 but it was Win95 that was the real beginning of the end of the MS/DOS ISV market. I was reading those monthly market share / vendor share channel stats from 1985 onwards. It was part of my job as someone who was running dev teams in exactly those segments. On those platforms. Mac /MS/DOS / Win 3.x
MS had weak market positions, basically an afterthought, in wordprocessors and spreadsheets until the platform change from MS/DOS to Win 3.x. Multiplan trailed Lotus 1.2.3/ Quattro etc. Word trailed Wordperfect, Wordstar etc. It was the Win platform change, and the huge effort MS put into using the platform change to destroy the competing products that first gave MS dominance of the office productivity software market.
MS were aided by the attitude of some at Lotus but the story put about that Lotus held back on porting to OS/2 / Win3.x is not true. The fact that Win 3.x was a piece of junk when it came to trying to write serious business software for it was closer to the truth. I know for one big name product of the time, who beat MS hands down for years in that market segment, the port to OS/2 was very straight forward, measured in weeks, while the port to Win 3.x took many months. Because the Win 3.x platform was so unstable. At least in that case there were no deliberate attempts by MS to hobble the product. Like with Lotus and others. If you know were to look in the source code you can still find the very suspicious code in Win16 code in the Win2k source code that is out there in the wild.
MS also spent a huge amount of money in the channels buying customers and locking out the competition. Some of it was alluded to in the Anti Trust case. It ranged from withholding MS/DOS or Win 3.x licensees from PC vendors who traditionally bundled competing products unless the competitors products were dropped to what was basically outright bribery in the wholesale channels when 5 companies accounted for 90% of all software sold retail in the US.
As for Excel. It was a "success" because it was the only product MS had which could actually run on Win 3.x / 95 in the early days with any kind of stability. The MS/DOS Multiplan code being such a complete dogs dinner. Even by MS standards. The only reason that Excel was kind of OK was because it had to run on the Mac without crashing every 5 minutes. The very first release did that. The MacOS enforced a discipline on the code which made it almost stable. The actual code itself was horrible when dissembled. The stuff that was not intermediate code fragments was very awkward glue code to the MacOS API traps. Easily the ugliest code I ever traced through with MacsBug.
So the reason why you have no viable commercial alternatives to Excel is because MS spent a huge amount of time and money making sure you didnt have any alternatives. Which is why their actual net profit on ever copy of Excel shipped is around 95%. Lucrative monopolies like that dont happen by accident you know.
-
-
Wednesday 7th October 2020 11:22 GMT Anonymous Coward
Re: I get it
> Aaah, I forgot the golden rule: Every situation in Tech over the past 25 years is because Micro$oft did something bad, not because someone else did something stupid.
Not quite correct.
The actual rule is - In every single situation involving MS they will be breaking the law in every way they can but the final death blow for almost all competitors was usually self inflicted. If the competitor shows even the most basic competence, and willingness to get in a bare knuckle fight with MS - no holds barred, the competitor will often prevail.
Lots of examples of both situations over the last forty five years.
There is no point bringing the Marquis of Queensbury Rules to a Cage Fight. You will lose. Every time.
-
-
-
Wednesday 7th October 2020 11:14 GMT Anonymous Coward
Re: I get it
> so.. you're saying that Lotus didn't fuck up Jazz and didn't kill Modern Jazz, eh? That they didn't wait
> until the 1990s to try to move 1-2-3?
> Right. It's all MS's fault. Sure it is.
So you obviously were not there at the time. In the business. You sound like a casual user not someone in the ISV dev business trenches. Modern Jazz was a MacOS product and MacOS was less than 3% of the spreadsheet market (by revenue at the time). Late 80's. A rounding error. End of story. Talking to the Mac team at the time I never got the impression it was anything other than revenue and potential revenue return on investment. The Mac was a bad bet at the time for business productivity software revenue. All the good R.O.I segments were creative content creation and educational. And a few verticals. No one else made real money on the Mac.
The Win 3.x port I was referring to was of a codebase that had been ported to OS/2 in 1987/1988 and I saw running on a pre-release Win 3.0 dev machine in spring 1989. At the time the product still outsold the MS product 2 to 1. There was a Win 2.0 version kicking around internally but Win 2.x was such a joke it was never ever considered for release. The guys in Cambridge, MA were doing exactly the same sort of stuff at the time with their product lines. I think their first OS/2 release came out in '89. But there again we did not have to deal with the guys in Redmond very deliberately adding code to the Win 3.0 codebase to crash our products. Much easier to do with spreadsheets.
Yeah, there was some hubris with some of the MS competitors but if you were in the business and followed closely the Anti-Trust case an all the other MS lawsuits over the decades (there are hundreds) you would know that MS would stop at nothing to destroy anyone they perceived as even the smallest potential competitor. As Jerry Kaplan at Go and Jean Louis Gasse at BeOS (to name but two) learned the hard way as they completely underestimated the criminals depths MS would stoop to in order to destroy anyone and everyone they perceived as competition. No matter how minor or peripheral.
MS was little better than an organized crime outfit in how it operated.
-
Wednesday 7th October 2020 13:53 GMT James O'Shea
Re: I get it
Modern Jazz was going to _start_ as a Mac product, to be moved to Win 3.x. It was abandoned. 1-2-3 for Windows and Mac was very, very, VERY late. It's not MS's fault that Lotus was late. And when Win 95 and NT came out, those who had Win 3.x and/or Mac applications up and running had an advantage to getting them on the new systems, so Lotus being late to Mac and Win 3.x bit Lotus _twice_.
And, oh... I recall there being swarms of word processors and databases and spreadsheets for Mac and Win 3.x. Excel was in the lead, but Word was not. And Access didn't exist. (Still doesn't on Macs; FileMaker Pro kicks its ass, and MS knows it and won't even try to compete.) Some of the other word processors (Nisus on Macs, for one) are still with us. Word Perfect came to Macs with lots of hype, and died a horrible death not due to MS machinations but due to massive incompetence on the parts of, successively, Word Perfect Corp, Novell, and Corel. I had a copy of Word Perfect for Mac, it did things that no other word processor, on any platform, was able to do at the time. Shame about how management screwed things up, starting with an insane marketing campaign and ending by _requiring_ users to _pay_ for certain updates (yes, plural) before being able to get the 'free' update to make certain features work with versions of Mac OS beyond 7.5.x.... and this after previous management had given those updates away... For a while WP for Mac was available for free from the vendor's site. Then they tried to charge for it. Then they just pulled it. It still worked into the early days of OS X, until Apple killed Classic, you just couldn't get updates and by that time other word processors had passed it by...
It's simply a fact that Excel wound up ruling because there sumply wasn't any competition worthy of the name. The big vendors ignored Mac and Win 3.x, and paid for it when 95 and NT arrived. Many small vendors who went to the small ponds of Mac and Win 3 are still around, making money, selling product; Lotus no longer exists as a separate company, and neither does Borland, Word Perfect Corp, and a host of other larger operations. It's not MS who killed them, it's their management.
-
-
-
Wednesday 7th October 2020 13:43 GMT Anonymous Coward
"the port to OS/2 was very straight forward, measured in weeks"
Sure, we all saw those OS/2 ports coming from everywhere.... porting a text based DOS application to any GUI OS was a task requiring a deep rewrite. If it had been just a matter of weeks, OS/2 would have had all the software it needed before Windows 3.1 had any chance to get it. Or did you mean a text-based port to OS/2? It was easier, but those had none of the advantages of GUI applications.
Sure - having two incompatible GUI OS posed a dilemma - which one will succeed? Obviously MS had far less issue because they cared about Windows only. IBM too didn't instilled trust when it tried to make the PC architecture proprietary through the Microchannel architecture. Those who had bet on Windows early like Adobe are still here.
Lotus didn't have OS/2 ports even after it was bought by IBM - I know because i tried to use Lotus SmartSuite instead of Office - but that was very ugly software. A lot of time spent trying to reinvent the UI instead of fixing bugs and making it useful.
Moreover even under DOS 1-2-3 was already suffering Quattro competition, since 1-2-3 lacked even graphics and laser printing layout - both requiring expensive add-ons - in a time when without LANs deployed everywhere prints mattered far more than today. And the time and resource spent to try to block Quattro could have been used far better.
-
-
-
-
-
Tuesday 6th October 2020 14:00 GMT jbstarbug
Alternative?
Whats the alternative to Excel then? Is there a simple bit of software that allows you to create tables for data classification and statistical analysis without having to know SQL or Python etc?
My wife is using Excel for some casework analysis (how many cases were apealed, overturned etc along with various reasons). She is not very familiar with it and when I have been helping her it really strikes me just how easy it is to screw up and Excel spreadsheet/pivot table.
-
Tuesday 6th October 2020 14:40 GMT vtcodger
Re: Alternative?
"Whats the alternative to Excel then?"
OpenOffice/LibreOffice I suppose, but they seem to me more or less Excel look alikes. Koffice, but I don't think it's maintained any more. Emacs org-mode tables, but they are not for the faint of heart. Personally, I was kind of fond of the spreadsheet in Microsoft Works, but I think getting it to run in a modern software environment might be, at the very least, challenging. And converting it's files to any other current format might be even more difficult.
-
-
-
Tuesday 6th October 2020 21:20 GMT Jan 0
Re: Alternative?
@bolac
It is very fair. Gnumeric is just a spreadsheet. Does anybody expect it to function as a database or a word processor?.
If you want to do statistics properly, use a Statistics package (for example Minitab).
If you want to create graphs, use a graphing package (for example Graphpad).
You can fill in the rest...
-
-
Tuesday 6th October 2020 15:33 GMT Flocke Kroes
Re: Alternative?
For something approximately single user I use CSV edited with a text editor for the data and python for report generation (reportlab) and emails. There are a mixture of costs and benefits.
The source data will probably arrive in multiple spreadsheets with fields in inconsistent columns. This can be converted to CSV an run through some python regexes to spot bad data, data in the wrong column, floating point phone numbers, duplicate records and all the usual mistakes caused by data entry into a spread sheet.
Update requires discipline to deal with commas inside fields and adding the correct number of commas when there are a few blank fields in a row but you can recycle your initial data correction tool to report these problems. (Update turned out to be completely beyond the ability of one trying-to-be-helpful Mac user who did not have/could not find a text editor or get a word processor to output in any format usable without whatever strange software she was using. [ended up with pdftotext and a script to get to CSV]) The good news is that problems are easily visible, machine detectable and human correctable without arcane knowledge of the guts of excel.
Next comes your first big payoff: automatically generated documents can have a consistently spelled names, addresses, emails and phone numbers for each job title. The entire document set can be automatically regenerated when any contact detail changes or when a different person takes responsibility for a job.
Basic queries can be done with awk | wc -l.
Mass snail mail can be handled with a PDF for the documents, a PDF with one envelope sized page per recipient and one of a number of companies that can combine the two and sort out postage.
Python has the libraries required to send emails with attachments. It is easy to add non-trivial logic to handle who prefers snail mail, who has expressed interest, who has already paid, output a list of who would receive what so you can check you got it right and send only to yourself so you can proof the output.
Python has excellent documentation but requires troublesome thinking skills. Reportlab has a barrier to entry but the PDF output will not display differently because someone has a different version of Word with the wrong printer driver selected. Taking the time and effort gives you valuable skills not tied to technological lock-in, spyware and adverts.
Finally on project handover some computer illiterate can import the CSV data into Excel and think they can do what you were doing because Excel is expensive software used by business professionals.
-
Wednesday 7th October 2020 08:03 GMT Frederic Bloggs
Re: Alternative?
It's deeply unfashionable, I know, but could I gently point out that perl was not only designed for this sort of job, but also has modules that, for instance, can read Excel spreadsheets directly. If you insist on doing the extra 'convert to CSV/TSV|SV' or whatever step, it has a module that can read that *intelligently* as well. Perl has probably the best regex engine out of all the mainstream languages out there for processing data and transforming it. It can export the result straight into just about any database that exists. If you are mad enough, you can write the result back into new Excel spreadsheet file :-)
Perl has an IMO undeserved reputation for being difficult to learn. But any programmer worthy of the name could learn enough to do this sort of job by reading "Learning Perl" and some judicious use of StackOverflow in a hour or three. This sort of task is a very good first practical program to write,
I cannot stress enough that the "Practical Extraction and Report Language" was designed for these jobs - the clue is in the name. And it's faster than Python.
-
-
Wednesday 7th October 2020 16:26 GMT Frederic Bloggs
Re: This/These
Certainly if you want to do complex things efficiently, say a real time world wide distributed messaging system with ~6000 concurrent users, then you need to dig a bit deeper. Or you simply display that important attribute of a programmer "hubris" and then decide, as I did, to just start writing that app to learn perl. After all: "how hard can it be?". Answer: after understanding some subtle concepts, and using lots of examples written by other people, "not that hard at all".
Another benefit is that this nearly 93000 line perl program (obviously split up into modules) written for linux/unix, runs on Windows with only about 20 extra lines (all to do with networking).
But for this job. Simple basic perl and some CPAN modules is all you need. I could probably write it in less than 100 lines (assuming no extra processing other than accepting lots of little spreadsheets or CSV files, some checks and adding them to a database). You will need to know that @thing is a array and %thing is a map - but then you need to know that for any language. None of the other things you mention would be required. Definitely no magic nor subtle concepts required here.
-
Thursday 8th October 2020 14:17 GMT Brewster's Angle Grinder
Re: This/These
Caveat: my perl is very rusty these days and out of date. So I may have screwed up the examples.
Some of why it was hard may be how it subverted my expectations as a programmer. I spent a lot of time struggling to work out which sigil to use. "
my ($x, @x); $x[0] = 4; print $x;" is a brain fuck when you're used to sigils being used to delineate the underlying type and not vary before your eyes (cf. Basic's x$ x@) And thenlocaldoesn't do what you'd expect. Maybe a newbie to programming would find it easier with a clean slate. But I found it harder than mastering C or asm.In terms of magic, I give you
chopand all those other "operators". And you're going to need references for the DBI. It does look as if open is now vaguely sane. (i.e. bareword handles and the aforementionedlocaldon't don't have to be mastered. And a three arg form mean you don't have to worry about magic filenames.) We're glazing over regexs but they're never far away and can render examples impenetrable and exactly what you might need to coral some ill formed CSV. In fact, this is example code from the chomp man page:while (<>) {
chomp; # avoid \n on last field
my @array = split(/:/);
# ...
}
That's why I love perl, but you need a good grasp of what's going on. And a lot harder to explain than the equivalent in most other languages.
Perl selects for people who are smart and willing to put in the effort. That's a good combination for producing good code. But I'm not sure its for a newbie and not where I would start them. And, alas, I would struggle to find an excuse to start a new project in perl these days.
-
-
-
-
-
Tuesday 6th October 2020 16:48 GMT Loyal Commenter
Re: Alternative?
Learn SQL.
Seriously, if the concepts of "tables", "rows/records" and "columns/fields" is too onerous for your wife, then she should just give up before she starts.
There is plenty of material on the internet on how to write SELECT, INSERT, DELETE and UPDATE statements. Basic SQL is very easy to pick up. There are also plenty of free SQL compliant databases. Yes, they might be a little tricker to install and configure than Excel (for instance the free versions of MS SQL Server, and indeed the paid for versions, aren't the friendliest to install), but half an hour spent with google should get you past any problems.
Then she can do it properly, and if she actually needs something to be in Excel, it can be extracted from a proper database easily enough.
Once you know how to use SQL (and have got past the inevitable mistake of forgetting the WHERE clause), you'll find there are far fewer pitfalls than there are with Excel.
-
Tuesday 6th October 2020 21:02 GMT Geoffrey W
Re: Alternative?
RE: "Learn SQL"
Don't be silly. The only way there are fewer pitfalls than Excel are if you look only at the pitfalls of Excel. SQL databases add a whole raft full of other pitfalls. And expecting a person who is barely coping with Excel (no Ms W, I am not looking at you...) to learn SQL and databases is like expecting cats to work the vacuum cleaner round the litter box; They wouldn't want to even if they could because...Why?
-
Wednesday 7th October 2020 02:55 GMT logicalextreme
Re: Alternative?
I think that you'd probably find quite a few people who use Excel inappropriately are in fact silently struggling with all the nonsense it brings, and have a head far more suited to databases. You can see the lights go on in their eyes when you explain the basics of the relational model to them. SQL is just the fairly-standard means of interaction with that model, and as a bonus it didn't get arbitrarily rewritten to be upside down and back-to-front with keywords in a different language and character set in 2007 (that's the Ribbon interface, for those playing the analogy drinking game). Learning anything never ends, but if somebody's realised that it's actually going to solve their problems, they'll learn under their own steam.
Crucially, the pitfalls of an RDBMS don't tend to be data inaccuracy — standard SQL and the way numeric types are stored can admittedly make them a bit shit at maths sometimes, but they don't go fucking existing data up just because you looked at it funny. The fact that Excel has the concept of a "format" and no concept of data types makes it fundamentally unsuitable for anything but the most basic, unimportant data tasks.
-
Wednesday 7th October 2020 10:02 GMT Primus Secundus Tertius
Re: Alternative?
@logicalextreme
"Excel has the concept of a "format" and no concept of data types".
Excel has major types: text, numbers, and dates. But its default is 'general', where it just
analysesguesses. General is the general cause of all the generally quoted faults in this article.
-
-
Wednesday 7th October 2020 09:34 GMT Loyal Commenter
Re: Alternative?
If you're doing any sort of data analysis, a SQL database is the appropriate tool. If you don't have a head that can understand databases, you don't have a head for understanding data, and you are doomed from the start.
Putting your data into a database allows you to properly organise it and analyse it consistently (Excel really doesn't and has a number of pitfalls to catch the unwary). It is the right tool to be able to store the data in a structure that represents its real-world structure (rather than a flat spreadsheet).
You might balk at the idea of using something like MS SQL Server, but if one is starting from scratch, there is probably a lot less to learn than learning to use Excel to get the job done.
You also have the benefit of it being transactional, and logged, so you can often undo your mistakes (if you know how), unlike with Excel, where a mistake can mean your data is gone, with no idea why or how.
-
-
Wednesday 7th October 2020 09:33 GMT ibmalone
Re: Alternative?
SQL would not be my choice for data analysis (see reply further down). For storing, sorting, extracting data yes, you can even do some of your data cleaning and preparation in it, and I'll confess to having done the occasional avg() and group by, but the statistical tools it offers are very limited and how are you going to draw a graph?
-
Wednesday 7th October 2020 10:24 GMT Naselus
Re: Alternative?
"Learn SQL."
Seriously, this.
If you have a problem that requires a database as part of the solution, then I'm afraid you basically need to learn how to identify and use a dedicated database system. Attempting to make your own in a glorified calculator because you can't be bothered learning to use the correct tools is like deciding to design your own car because you can't pass a driving test - it's probably harder than just doing it properly, the result you come out with doesn't solve the actual problem (your inability to drive safely), and the fact that you don't understand how to use the existing solution suggests your own is almost certainly not going to be better than the ones real experts have been refining for 100 years.
Learn to identify the right tools for a job, and then learn to use those tools. The reason for this track'n'trace fuckup is a complete failure on both counts.
-
-
-
Wednesday 7th October 2020 13:26 GMT Anonymous Coward
Heretical statement
I used to be an Access evangelist - great tool for prototyping databases and a good way to set up user friendly GUIs for data entry and reporting. But over the years I've fallen out of love with her. Her habit of changing her storage formats on every version upgrade. The way anyt programming will be borked by some random update to a Windows .dll you never heard of, the dumbed down stupidity that she can't read her own old formats - let alone useful ones like .dbf. And before anyone screams, .dbf files store tabular data in ESRI shape files, which are the lingua franca of the GIS world. If I want to read tabular GIS attributes to or from Access the simplest intermediary is...fanfare......Excel!
Anyway.- sqllite for storing stuff, R for everything else!
-
-
Wednesday 7th October 2020 02:58 GMT logicalextreme
Re: Alternative?
Graphical ETL, in my experience, is as bad as Excel if not worse. The more fluff you put on top of things to abstract people away from the data and the reality of what they're doing, the worse the data that comes out will be. I've reserved a special place in hell for SSIS, which in its defence is technically a free gift.
-
Wednesday 7th October 2020 09:29 GMT ibmalone
Re: Alternative?
Stata is the most welcoming of the stats packages I've seen, simple tasks can be done through the gui, you can do an entire basic analysis without typing (and generate the commands used at the same time, so the analysis can be stored for reuse independent of data). It's not cheap though. SPSS can also be used from the gui and isn't cheap either, but they've been teaching psychology students to use it for years, so you don't need to be particularly into programming to make progress.
R with R studio has a bit more of a learning curve, but the open source version is of course free. You can view data, graphs are plotted in a window. Yes, you'll need to learn a little R, but it cuts out much of the boilerplate you need to do analysis in a generic language. It has its own idiosyncrasies that make it a little awkward for people from a more traditional procedural background. Strongly recommended to start with the tidyverse framework (includes dplyr, ggplot2 and others) which make it much more powerful. Here's a quick start https://www.datacamp.com/community/tutorials/tidyverse-tutorial-r
There is a learning curve with all of these, as you have to think in a more structured way (a bit like going from a word processor to latex/lyx), and excel will get you a lot of the way for simple noodling around with simple sets of numbers provided you're careful, but with a clear defined problem to work on people can often manage to get things done with these kind of tools more easily than with a general task like "learn python".
-
Wednesday 7th October 2020 09:41 GMT macjules
Re: Alternative?
It all depends upon the job you need to do. For something like your wife's casework analysis then I should think Excel is ideal. If you want to run a check on 65m detailed rows of CSV data then I suspect that you might prefer to use a tool such a CSV Linter, which strangely enough HMG actually built for nationalarchives.gov.uk to check large CSV files. I wonder why they didn't use that for PHE?
If you are a financial institution and trying to run calculations on fractional percentage trades within microseconds then something written in Python is probably going to be far more effective than Excel, or any other spreadsheet software come to think.
-
-
Tuesday 6th October 2020 14:01 GMT Quenda
What should I use instead
Ok we should not use Excel. What should a non programmer use instead? If you are going to destroy the city you need to build something in its place. And it needs to be as easy to use, as powerful and not require a small army or programmers or consultants to maintain.
-
Tuesday 6th October 2020 14:47 GMT Andy The Hat
Re: What should I use instead
In the late 80's one of my first newbie tasks was to move a database off a Harris mini onto a PC and I used a simple DOS-based flat file database application that was easy to set up, could search by example, generate standard or custom reports with a few keystrokes, export data etc. Worked absolutely fine and ran off a floppy! That system worked really well for years until some IT department big-wig decided it needed migrating onto Access at which point complexity and lack of usability killed it totally for the users. After a while it got migrated onto Excel where, to be honest, it was horrible and difficult but it worked *for the people that needed it*.
If I want to do the same initial migration now the words to be spoken will probably include "Access", "SQL" and "Oracle", "cloud", "hardware upgrade", "internet stability" ... And, when the punter is scared enough of the prices, horrendous learning curve, cost of books and cost of courses and realises 90% of their investment is actually for useless software components because all they want is a simple flat file database that's easy for non-IT geeks to drive, they'll end up using Excel (probably badly) as there's bugger all else and they 'know' it ...
Perhaps, for the average punter, we need to take a step back to the database past and simplify so the punter has an alternative to Excel? But just think of the money we'd lose if people could actually do what they wanted to *well* with minimal input from consultants, support desks, training courses, books ...
-
Tuesday 6th October 2020 14:58 GMT gormful
Re: What should I use instead
How about just Excel, but with a flag that turns OFF all automatic data conversion? Just edit the strings that are there; don't strip leading zeros from stock numbers or interpret gene names as dates.
It would be AWESOME simply to have an application to edit data in CSV files that DOESN'T "re-imagine" data values whenever it bloody well feels like it.
-
Wednesday 7th October 2020 07:36 GMT Andy The Hat
Re: What should I use instead
"don't strip leading zeros from stock numbers "
Sorry, I'm sure all modern users are aware of the stupidly obvious Lotus 1-2-3 formatting command sequence - user realises what has happened, user swears, user edits number with leading apostrophe - to prevent this? I can't believe I still have to routinely use that 'technique' ...
-
-
Wednesday 7th October 2020 09:09 GMT Andy The Hat
Re: What should I use instead
Actually you are incorrect.... try entering a leading zero number into a default Excel cell and it truncates the leading zero ('cause that's what it does) - if you then change the cell format to TEXT the leading zero is not there, the entered data has been parsed as a number, truncated and AFAIA the leading zeros are unrecoverable data.
The last thing a single user of a multi-user sheet wants to do is find that truncation happens on a single cell entry, take the sledgehammer approach of changing the entire column format (to text or whatever) to guarantee it never happens when actually it may have already done so ...
-
-
-
Tuesday 6th October 2020 16:34 GMT Doctor Syntax
Re: What should I use instead
"Ok we should not use Excel. What should a non programmer use instead?"
Email or the telephone. To ask a programmer to put together a real application.
For a lot of purposes the spreadsheet is good enough but at the very least you need a sanity check on the results and sometimes "good enough" isn't really good enough and you have to require better. Dealing with people's health and lives is one of those times.
-
Wednesday 7th October 2020 13:34 GMT Anonymous Coward
Re: What should I use instead
Hello - Programming department? I need a widget. What's that? I don't need a widget because we already have a EPS (Enterprise Widget Platform)? I know but it doesn't do the thing I need it do. Can you add that functionality? Change request? OK. So when can you do it? Next year? But I need it now. I have to take any request for priority to the change request committee? When does that meet? Next year!!!. OK How about if I buy a Widget from Widget's-r-us? It needs to be evaluated by IT before it goes on a corporate system? When can you do that? Next year????????. Forget it.....scrabbles around the start menu for copy of Excel.
-
-
Tuesday 6th October 2020 16:49 GMT cyberdemon
Re: What should I use instead
I think they should be teaching GNU Octave in schools instead of Excel. Or better yet, teach "How to make a database in Python". Django is a nice starting point! Very simple to make a good database with minimal code.
The problem isn't "what should a non-programmer use" but instead "why are there so many people who call themselves non-programmers"
It's a bit like asking "what type of boat should a non-swimmer sail?"
-
Wednesday 7th October 2020 09:47 GMT ibmalone
Re: What should I use instead
Ok we should not use Excel. What should a non programmer use instead?
Are they writing macros? Are they creating calculations that depend on other calculations? Then congratulations, they're a programmer already.
If you run out of memory slots in your calculator, need to knock up some graphs or do some quick data entry that isn't big enough to warrant setting up a database (and doesn't have horrible GDPR implications), then excel will do. If the task has gotten bigger and you've got sheets that depend on other sheets and calculations that need to be filled down and data that needs to be pasted in and a spreadsheet named quarterly_report_v2.2_2020-q3-tmp-newmacros.xls, then stop and think what you're actually trying to do. Maybe that's what should be used instead, some introspection.
-
Wednesday 7th October 2020 10:46 GMT Naselus
Re: What should I use instead
It depends what you're doing. Are you doing something that a spreadsheet is actually for? Then contine using Excel. Are you doing something that a spreadsheet is not actually for? Then go and find something that is designed for the thing you're doing and stop blaming Excel for not being a word processor/database/first person shooter/Boeing 747.
Spreadsheets originated in accounting as basically manual calculators. Computerized spreadsheets are, at their heart, automatic calculators - much like, y'know, actual desktop calculators, only generally more feature-rich. That feature richness has led to people forgetting that what they're looking at is basically still a mechanism intended to facilitate doing fairly small numbers of relatively basic 2 or 3 decimal place calculations.
Use it as a calculator and you're generally ok (not for hyperaccurate statistical stuff, but few people outside the sciences need that and those that do know not to use Excel generally). Use it for anything else - storing vast amounts of data long-term, or just as a table-making device or what have you - and it will probably let you down. Yes, you can use Excel to code a game, or to make art, or to store large amounts of data. You can also cook your Christmas turkey with a blow torch, but it's not really the right tool for the job.
-
-
Tuesday 6th October 2020 14:22 GMT mark l 2
While there are times when an Excel spreadsheet is perfectly adequate for the job, it sounds like using it for contact tracing, with what has to be tens of thousands of records, if 16000 can get lost without anyone immediately noticing is clearly not one of those times when it was the right tool.
-
Tuesday 6th October 2020 16:34 GMT cyberdemon
CSV import?
Has anyone asked why in gods name did they have a CSV file with 32k+ "columns" in the first place?
CSV itself is really not optimal for "large" numbers of columns. I think this farce goes beyond simply abusing Excel. Clearly the task was given to some poor overworked underpaid clinical admin clerk, without anyone realising that they are dealing with a large scale data management problem that should have warranted the services of an experienced database engineer.
"using a nutcracker to demolish a wall" springs to mind.
-
-
Wednesday 7th October 2020 15:24 GMT Brewster's Angle Grinder
Re: CSV import?
The column limit on .xlsx is 16K so that initially sounded the right size. But apparently there were multiple lines per record which is why the .xls 64K row limit came into play. I hate to think what this data actually looked like or how they got it from a CSV file to multiple rows per entry. (Did every column in the CSV get turned into a row with blank rows between?)
-
-
-
Tuesday 6th October 2020 14:26 GMT big_D
Its worse than that...
When I was "young", I worked on a sales forecasting system which downloaded data from a VAX Oracle database and performed calculations in Lotus 1-2-3. The project manager wrote a quick-and-dirty prototype in 1-2-3 and presented it to the customer as a set of "working" screen mock-ups. The finished project should have had a database and use C++... Only the customer said, "no, 1-2-3 is great! All our sales people have that already, just get the prototype working!"
I was called into the project at that stage and no amount of wailing helped. The customer was adamant. So, we expanded the 1-2-3 model. It ran in DOS, it had around 40 sheets it loaded in one after another and ran calculations... Then it started doing funny things.
Self-modifying /-code macros didn't help (dynamic cell references, as 1-2-3 didn't have variable). It worked fine in debug mode, stepping through the thousands of lines of /-codes. But actually run it? It fell over randomly and gave the wrong results. After tearing our hair out, we actually contacted Lotus support. They asked for a copy of the spreadsheet, they got a 2MB bundle of tables (we are talking DOS here, 640KB main memory, 1MB with Himem.sys and a 40MB drive!).
They looked at it. They wept. Their official answer was, "forget it, 1-2-3 was never designed for anything this big or complicated!"
-
Tuesday 6th October 2020 14:42 GMT Anonymous Coward
When you need something fast without the hassle of creating and hosting a database, access, permissions ,defined structures then excel is your go to albeit it should be your temporary go to. You can't email a database or chuck it on a network drive for someone to access however every one can get into a spreadsheet. In the case of track and trace they needed something quick that could work with all the various companies and individuals doing the track and trace and enable work flows from said data. Why they didn't move it from Excel is beyond me especially the amount of money thrown at it. The way I see excel is that it doesn't need to get better because it does what it's supposed to with spreadsheets/data. Now if someone comes up with an easy to use/setup database (I'm talking excel user easy and universal software wise) they might be onto a winner.
-
-
Tuesday 6th October 2020 19:42 GMT Yet Another Hierachial Anonynmous Coward
Why they didn't move it from Excel is beyond me.....
"In the case of track and trace they needed something quick that could work with all the various companies and individuals doing the track and trace and enable work flows from said data. Why they didn't move it from Excel is beyond me especially the amount of money thrown at it."
Remind me again, who is head of track and trace? And her computer orientated qualifications are?
News next week - USB stick with spreadsheet of 16000 Corona patients found in back of taxi/bus/pub......
-
Tuesday 6th October 2020 20:56 GMT SGJ
Maybe 'something quick' was required in the first few days of Track & Trace. Maybe. As I have seen many times in my 40 years of working in IT (first as a developer and then IT management) the 'something quick' became 'the way we do it'. There is simply no excuse for still using a spreadsheet for such an important task months later.
-
-
Tuesday 6th October 2020 14:52 GMT Anonymous Coward
VBA Security Security Security Security... I can't hear you!!!
This sounds more like a FOTW than an El Reg article. Amusing, makes some interesting points, but really belongs in Bootnotes.
All this really boils down to "A fool with a tool is still a fool".
Excel is great at what it does - really. It has simplifed or speeded up my job no end on occasions. It has faults and limitations (some very basic and silly), but I've not used a tool that hasn't.
And yes, an excel spreadsheet is likely more secure than a database set up by someone who has no idea what they are doing. After all, it is rather difficult to locate a file with a simple port scan...
-
Tuesday 6th October 2020 16:20 GMT Joe W
Re: VBA Security Security Security Security... I can't hear you!!!
Excel is for two columns, one for money in, the other for money out and comparing those. That's it. Using it for anything close to production with the amount of data and the criticality in this most recent example is at least.... questionable.
-
Tuesday 6th October 2020 20:24 GMT Glen 1
Re: VBA Security Security Security Security... I can't hear you!!!
Given the propensity for JavaScript libraries dedicated to graphing and similar, do you (the el Reg readers) think the move to web-based stuff is an improvement on Excel?
I suppose having a JS front end implies there is a proper database somewhere at the back...
-
-
Tuesday 6th October 2020 16:34 GMT Doctor Syntax
Re: VBA Security Security Security Security... I can't hear you!!!
"And yes, an excel spreadsheet is likely more secure than a database set up by someone who has no idea what they are doing."
And when the spreadsheet is set up by someone with no idea what they're doing you get the mess we're encountering now.
-
Tuesday 6th October 2020 14:58 GMT odyssey
Blame the user not the tool
This reminds me of Bjarne Stroustrup's observation that there are two kinds of programming languages - the ones people complain about and the ones nobody uses. Excel is a good general-purpose tool for what it's meant for. There are lots of horror stories because it's so widely used. Any other tool that was as popular would have people misusing it. Most of the horror stories listed are human error.
Having said that, Excel is not meant to be used as a database (why MS makes Access and SQL Server). It's not MS or Excel's fault this happened, it's people stupidly importing csv files into a spreadsheet program when databases are the right tool for the job and freely available.
-
Tuesday 6th October 2020 19:28 GMT Nevermind
Re: Blame the user not the tool
Yes, or you work in a big corporate with enough clout to refashion all your textual forms as Excel spreadsheets because that's the only tool you know how to use for easy layouts (why am I thinking of chimpanzees and sticks here?)...Word obviously not your cup of tea, PDF forms completely non grata...."ah excel" you go and you even password protect it. When the user offers to make it better (read usable), just say "Well, no one else has complained so no"....
Looking at you RR, you cockwombles.
-
-
Tuesday 6th October 2020 15:03 GMT katrinab
Anyone Remember Lotus Imrov
No not Lotus 123, of course you remember that if you are old enough.
Lotus tried to fix the problems with 123, which are the same problems Excel has today.
Lotus Approach was also not bad if you wanted something a bit better than a spreadsheet but not as complex as Access. People would have had that on their Smartsuite CD back before they switched to Office, but they never used it.
Microsoft has PowerBI which seems to be trying to do the same thing. I haven't looked at it much, and probably nobody else has either.
You can try to make better things, but people will continue to misuse spreadsheets because that's what they know.
-
Tuesday 6th October 2020 20:40 GMT Krassi
Re: Anyone Remember Lotus Imrov
Upvote for Lotus Approach. Back in the day it helped me do simple data handling from a starting point of knowing squat about databases.
But the main article misses big advantages of Excel. It is flexible, easy to adapt - using it is the ultimate in agile programming, do something, change it , change it again, keep on keeping on. There are lots of real world uses that don't have large data bases; that don't stretch desk-top computing power even passed through excel and VBA; projects that don't have the time or budget to bother "real" programmers; jobs that are a little different each time. Every one has excel, so different organisations can work together.
Ironically, excel was probably being used in this case because two "proper" software products couldn't interface AT ALL.
Anyway, Excel is approved by the National Institute of Standards and Technology ! Have a look at this great example of excel VBA
https://pages.nist.gov/AGA8/downloads/AGA8.XLS
-
Wednesday 7th October 2020 08:15 GMT Richard 12
Re: Anyone Remember Lotus Imrov
No, XLS was used because the receiving end was run by a complete and total idiot.
They use Excel to merge together multiple CSV files provided by the labs, then imported said abomination into the Trace part.
This is Dido Hardings fault, and she should be held accountable. Of course, she's a Tory
donordarling and so is instead being given an even more important job to kill people with.
-
-
Tuesday 6th October 2020 16:02 GMT bolac
All Spreadsheets Are Not Equal
It is not fair to say that all spreadsheets are equally shit. For example Gnumeric, the one created by the Gnome folks in two weeks as a demo app, was very popular among scientists because it calculated accurately and mathematically correct. For a decade, it was also a role model for good C programming.
-
Tuesday 6th October 2020 19:13 GMT katrinab
Re: All Spreadsheets Are Not Equal
It is still a spreadsheet though, and most of the problems that cause 90% of Excel spreadsheets to have errors will manifest in Gnumeric, because the problem isn't with the software itself, but with the fact that you are using a spreadsheet instead of the correct solution for your problem.
By the way the bottom right cell in my copy of Gnumeric (1.12.46 from the FreeBSD Ports Collection) is IV65536, same as Excel 2003. So it would not be suitable for use as a contact tracing database for exactly the same reason that Excel 2003 isn't.
-
-
Tuesday 6th October 2020 16:33 GMT jo so
30 years ago I remember an office department manager asking me, a programmer, how to improve his assistant's productivity. Every morning his assistant would log into a commercial off-the-shelf software program to run a bunch of reports to manually input into a spreadsheet to help measure sales, commissions, and goals. I analyzed the process, wrote some COBOL or RPG II program on a DEC VAX, and presented a new daily batch procedure that the assistant could run instead. My five hours of programming saved an hour each day of the assistant's time so that the assistant could spend more time on other needs. Those were the days.
-
Tuesday 6th October 2020 16:34 GMT Doctor Syntax
Step back here and ask a few simple questions (the answers might not be simple).
1. What data and in what format is received from the testing centre to accompany the samples?
2. What format do the lab instruments export?
3. What needs to be done to interpret the results from 2 and marry it up with 1.
4. How many systems need to be fed from this?
5. What format(s) are needed from that?
6. What needs to be done to prepare the output from 3 into 5.
7. What sort of scale is needed?
8. What's needed to accomplish 6 at 7 reliably and at appropriate speed?
9. What's needed to oversee the process and ensure its all running properly?
At that point you have a set of requirements for the processing needed. "A spreadsheet jockey" isn't likely to be the answer to 8 although it might e the answer to 9.
-
Tuesday 6th October 2020 16:40 GMT Neil Barnes
Forgive me for I have sinned...
I have used excel to automatically generate the C-code for a stinkin' great state machine.
I'm sorry. I'll try not to do it again.
But at least it's in the revision system, so if someone buggers it up with an accidental drag'n'drop, we can at least get back to something working.
-
Tuesday 6th October 2020 17:15 GMT c1ue
The OP is angry at Excel.
The OP apparently would prefer everyone use some specialized tool or language where the operations are utterly opaque.
Excel - you can see and trace everything going on. Not so for "modern" platforms.
Yes, it isn't scalable for Big Data or even medium Data - but it never pretended to be.
Being angry at something which can do so much, well enough, that it is pushed to the limits and beyond is silly since this applies to literally everything in IT.
-
Tuesday 6th October 2020 21:06 GMT SGJ
I have seen spreadsheets so opaque it was impossible to figure out what they were doing. No naming of ranges or cells, no documentation, links between Excel files, no understanding of relative and absolute addressing, columns in which all cells contain expressions - except for some 'special' cases.....
It is possible for a competent developer to produce a good spreadsheet except a competent programmer wouldn't be using a spreadsheet in the first place.
-
Wednesday 7th October 2020 03:14 GMT logicalextreme
I think that it is pretending to be though. More and more and more features get thrown into it. The row limit was upped from a perfectly sane 65,536 to over a million. The column limit was upped from 256 to over sixteen thousand. Where before people would have stopped, taken a step back and asked if it was the right tool for the job (mostly…some might end up partitioning over multiple sheets before they start wondering, and there's always the odd person that can't think for themselves and never even considers that a review might be in order), there are now 17 billion cells up for grabs on just a single sheet of a workbook.
If we can get angry at firearms manufacturers and tobacco companies for being enablers, we can sure as hell get angry at Microsoft.
-
Tuesday 6th October 2020 18:45 GMT Peter X
Row / column limits
Going out on a limb here, but maybe the row/column limit on Excel should be far far far lower? E.g. 100?
Then, people would quickly realise, it isn't the tool for the thing that they think it's for. And hopefully they'd then, correctly, use an RDBMS.
Oh, and after a user has watched YouTube videos and (maybe) checked the help information on how to use a pivot table, when they actually try to choose Pivot Table from the menu, it should pop up a message saying "You know this is not some magical thing; it's simply grouping and aggregation which is total piss to do with an RDBMS".
Then - perhaps - they'd know!! They'd understand!!!
-
Wednesday 7th October 2020 03:23 GMT logicalextreme
Re: Row / column limits
A thousand times yes. People's ire was directed at the Ribbon in Excel 2007, when it should have been at the limits.
I've come to assume that Microsoft knew that they'd be able to make more from per-user Office licensing than SQL Server core licensing, pricey as it is, because previously there seemed to be a perfectly obvious progression when you hit the more obvious limits of the software.
-
-
Tuesday 6th October 2020 20:08 GMT Anonymous Coward
I don't think the problem is Excel
I think the problem is stupidity, massive educational failure, or both.
I mean, I'm sure Excel is adequate at the things spreadsheets are adequate at[*], the problem is people using it for deeply inappropriate tasks with no understanding of the issues around that at all. I worked somewhere where people knew not to use Excel in this way. So they stored a huge and critical system configuration in a google sheets document with no version management at all because this was, obviously, better. But not better enough: the thing was obviously too hard to use due to the whole having to be logged in to your google account thing ... so they turned that all off. It was world writable in a sense which is hard to imagine: if you had access to a reasonably modern web browser and an internet connection you could edit it, with no need for that whole annoying 'authentication' or 'having an account' thing. I should have a look sometime to see if it is still there and if my little test edits are still in it. This was less than five years ago, And yes, of course it was government-related. And of course they did fuck all about it when I told them.
[*] Ok, I'm sure it's not. But if it was it would not help deal with the real problem, which is people.
-
Tuesday 6th October 2020 21:18 GMT theOtherJT
Re: I don't think the problem is Excel
The problem here as I see it is that like a bunch of microsoft products it does a ton of things it has no business doing. The correct response should be "piss off that's not what this is for" but for some reason every time someone says "Hey, I want to do <pick a thing> with Excel" microsoft just decide that implementing the most half-assed possible way of doing that thing is the right answer as opposed to saying "No."
See also the fact that Word is for some reason a web browser, and that Outlook does.... sort of everything that outlook does.
-
Wednesday 7th October 2020 10:30 GMT Primus Secundus Tertius
Re: I don't think the problem is Excel
I find it jolly useful that MS Word can read html files and turn them into docx (or pdf or txt or even odt).
But I agree with you about Outlook: it seems aimed at people who are pretending to be businessmen.
Libre Office Word can even produce and read back what it calls docbook. But other xml software I have says it is well formed but not valid.
-
-
Tuesday 6th October 2020 21:54 GMT Chris Miller
Re: I don't think the problem is Excel
A big +1 for education. 25 years ago, I was responsible for 2,000 desktops transitioning from Win 3.1 to Win95. We had three staff dedicated to training, going round the country teaching staff how to use Windows, Word, Excel etc.
There's an assumption that you can take a work experience bod, give them a copy of Excel and they can just get on with it. And the problem is that they probably can produce (after an unnecessary amount of trial and error) something that resembles a working spreadsheet. But they won't know that if you've a table of financial figures summed by row and column (imagine sales numbers divided by product and calendar month) that you should compare the sum of the sums of rows to the sum of the sums of columns, and if they don't agree something has gone wrong. And that's what education teaches you.
Instead we get moaning Minnies like the author of this article saying that Micro$oft is crap, when really the problem lies with management cost-cutting.
-
Wednesday 7th October 2020 08:50 GMT Aging Hippy
Re: I don't think the problem is Excel
The fact that Excel is the wrong tool for the job is irrelevant here. Hasn't anyone heard of reconciliation totals. At its most primitive:
number of tests received at centre = number of tests sent = number of tests done
Either the person "designing" the system had no idea or, more likely, their manager thought it was a waste of time.
Punishment should be to be taken back to the 60s and made to feed several boxes of 2000 cards into a card reader which rejects 1 in 100 while the PFY drops several handfuls onto the floor. Reliability breeds laziness.
Excel does have a purpose here - it's presenting the reconciliation totals for all files received.
-
Wednesday 7th October 2020 14:05 GMT c1ue
Re: I don't think the problem is Excel
The problem is that Microsoft's Excel programmers did a really amazing job of integrating all manner of extra capabilities into Excel.
You can query web pages. You can move data back and forth between sheets and even documents. You can bend, fold, spindle and mutilate cells and their contents.
That's the real problem: if there isn't a 100% perfect program to do something, it is easier to do it in Excel than to do "real" programming to create something custom.
So the real complaint is about Microsoft Excel/Ajax programmers being too good at what they do.
-
-
Tuesday 6th October 2020 21:05 GMT Trubbs
Golden Goose
I have paid a considerable portion of my mortgage supporting business critical full blown applications hosted up the back end of excel workbooks. They have just one big button to activate and everything else is VBA forms etc. The original developer left the company and as soon as I let slip I knew how to fix them I was encumbered.
Migrating/ rewriting to 'proper' desktop apps paid some more mortgage.
So I have a personal grudging soft spot for excel. Don't get me started on fsckng msaccess!
-
-
Wednesday 7th October 2020 02:29 GMT myithingwontcharge
Re: Disappointing
"It a little frustrating that el reg think excel is the story here rather than “if you starve public services of cash for 10 years don’t be surprised if their systems are held together with sellotape and string”
They claim to have spend something like £12 billion on this broken bollocks. Money is not and never was the issue, it was a total lack of skills and clue. I mean they appointed Dido Harding to run the thing FFS. If that doesn't tell you that it was never expected to work and someone's just pocketed the money, nothing will.
-
Wednesday 7th October 2020 11:24 GMT Dave314159ggggdffsdds
Re: Disappointing
Maybe because the nonsense you state is completely counterfactual? "Austerity" was just Tory propaganda Labour were stupid enough to think would benefit them, and so didn't push back against.
What we actually had, and still have, is a social care funding crisis. Austerity let them escape dealing with it, but it wasn't real.
-
-
Tuesday 6th October 2020 22:34 GMT Sparkus
Once Upon a Time.....
The end-user agreement and T&Cs for Netscape, Mosaic, and first gen MSIE had requirements something like "you will not use this software to control or monitor any equipment or process that could pose a danger to human life including nuclear power plants, medical life support equipment, and military weapons systems."
Was the subject of much lunch room laughter at the time.........
-
Tuesday 6th October 2020 23:35 GMT Blackjack
I freaking hate Excel
I would rather relearn Basic to make a small program that shows the data the way I want, haven't used Basic since the 90s, that deal with Excel unless I absolutely have to. And even then I can point the last version of Excel I used was from 2001 and bribe someone else to use the freaking new Excel for me.
-
Wednesday 7th October 2020 01:48 GMT Anonymous Coward
There was an alternative
Lotus re-invented the spreadsheet with Improv. I bought it and it was fantastic. Unfortunately, it did things differently to 1-2-3/Excel/Visicalc so didn't take off. Much better, but different. You had to learn a new way of doing things. Formulae had real names instead of letter-number. You could have multiple dimensions and immediately change your view of your date, just by dragging a dimension tile to a different location. I loved it.
This is the reason Excel is as it is. People are used to how spreadsheets worked in the past and don't want to learn a new product concept, even if it is a much better way of doing things. Microsoft could come up with a vastly improved spreadsheet concept, but if it works in a substantially different way to Excel, most people won't buy it. Lotus tried and got burned.
-
Wednesday 7th October 2020 08:41 GMT StewartWhite
Don't use the wrong tool for the job then blame the tool
It's easy to blame Excel (and tbf it's got plenty of stupidity inbuilt) but this is just a highly public version of using what can be a reasonable tool for the wrong job.
In a previous role I had the Accountancy team complain that their documents weren't as easy to type/use as everybody else's. Turns out they'd mandated that only Outlook and Excel were allowed to be used in their team so they weren't allowed to use Word.
You can use a hammer and chisel instead of a screwdriver to undo a screw but if you do, Shirley it's you're fault and not the hammer's?
-
Wednesday 7th October 2020 08:42 GMT ForthIsNotDead
A bit over the top
I've been using Excel for decades. With VBA I might add.
I typically use it to gather together configuration for SCADA projects and then have VBA spit out the SCADA config files (XML etc) at the click of a single button. Easy. Error-free, and the spreadsheet can be a single point of truth - at least in the initial stages before the SCADA system itself becomes the SPoT. Been doing it like that for decades.
Over the years I've seen a lot of spreadsheets that should have been Access projects, but everyone knows Excel, and nobody wants to learn Access. That's a shame because it's a decent tool, too. They're especially powerful when used together, when Access holds the data, and Excel is used to present the data.
Excel is a tool. If you use it for the wrong job then blame yourself or the people that are forcing you to use it for the wrong job. Don't blame the tool.
-
Wednesday 7th October 2020 08:48 GMT jim.warwick
Was this Excel or just Excel file format...?
What I read here is that the data clean up process used an out of date Excel *file format*.
This to me sounds like it could be a data clean-up tool that is using an old excel file format as it's output to be imported into the national database thingamyjig (whatever that is).
Working off a deprecated Excel file spec as your data interchange mechanism is not much better, but it's at least not using some hacked-together Excel macros...?
Would like to know the details...
-
Wednesday 7th October 2020 09:08 GMT Paddy
Shiny Shiny
If a company porposefully bakes its known "1900 is a leap year" bug into its Office open XML "standard" , then shame on those who ratified the standard and shame on those who entrust their data to a company who flagrantly ignores its bugs rather than fixing them.
Spreadsheets are as attractive as sirens; but more perilous!
-
Wednesday 7th October 2020 12:50 GMT Binraider
Excel is, what it is, and personally I don't blame the program for it's known limitations and difficulties when mis-used.
The bigger problem is corporate red tape associated with getting something better - personnel training, the myriad choices, ongoing support and so forth. There are no shortage of suitable tools; but cost involved in actually putting them on the ground for anything more than isolated instances just doesn't fly.
Cross compatibility of stuff is the key to getting out of the isolated instances problem, and SQL did, for better or worse, try to create a route to that for relational data. If one could create cross compatibility of expressing analytical relationships, inside a computer system, you would be onto something. Oh wait, that's called a programming language.
And once again, we are back to the skills / cost of deployment issue.
This headache is not going away, until we can instruct computers to build things of their own accord ala Star Trek.
-
Wednesday 7th October 2020 13:31 GMT Andy Denton
The problem is....
...even if Microsoft were to come up with a spangly new replacement for Excel that was perfect in every conceivable way, people wouldn't use it because It's Not Excel and Excel is all these people have been used to using for decades. Many people (accountants etc) that use Excel a lot aren't adventurous types and there's no way they'd trust their precious data to some new spangly product when Excel is still available.
-
Wednesday 7th October 2020 14:35 GMT Anonymous Coward
Wrong blame
Blame the tools who use the wrong tool, not the tool being used.
Excel is more complicated than a simple spreadsheet needs to be and I've never met anyone using a significant proportion of its capability. But, in the right hands, it is a good tool. The problem is that it's in the wrong hands.
I first encountered spreadsheets with Lotus Symphony and had to learn its quirks as it was being used as a database central to a multi million $ project. It had its limitations and these were accounted for - it was only pushed as far as it could be relied upon. Excel is far more capable but, as others have said, it's not a multi-user database. If I had to highlight one fault with Excel, above any other, it's that it's too easy to get out of your depth without realising it.
-
Sunday 11th October 2020 21:52 GMT MachDiamond
Re: Wrong blame
"Excel is more complicated than a simple spreadsheet needs to be and I've never met anyone using a significant proportion of its capability."
I've found it to be more complicated than it needs to be and then falls on it's face when trying to do really complicated stuff. Maybe it's awesome for accounts and lobotomized MBA's, but it's a PIA for engineering. The only reason I use it at all is compatibility or it wouldn't be loaded on my computer. For the places it doesn't work, I output a PDF from something else to communicate data to somebody else. I rather like that they can't mess with it.
-
-
Thursday 8th October 2020 11:11 GMT L'Ecossais
Compatibility mode
It seems to me that the cause of the problem here is very simple. Excel can in some circumstances default to "compatibility mode" which has XLS as the default file save. To resolve it, you have to set the default file save type in File/Options.
As to the error message when a Save is attempted, it is likely that the import and save were automated and any error message potentially logged. As the situation came to light on a Monday, what are the odds someone came in to work, viewed the logs, or even got an automated email, and had the Oh Sh!t moment.
-
Thursday 8th October 2020 11:55 GMT Torchy
Microsoft Money.
My Wife used Microsoft Money (the version that came with DOS 6.2) in her business up until two years ago when her accountant told her that she had to use Sage.
She could get a turnover of £720,000 accounted for down to the last ten pence.
That Sage software is so inflexible that the accountant is happy if it is to within £100.
When it is not broken do not fix it.
-
Friday 9th October 2020 17:49 GMT Anonymous Coward
Excel is Hell, but my IT support was downsized & laid off years ago!
Don’t rain on our parade. We are stuck providing intelligence to the unknowing using data dumped into spreadsheets by that one guy left in IT. I always start my conversation to those leaders with “this shouldn’t be done in Excel”, but they are more interested in the color used in the final Excel crunched charts than long term data accuracy and maintainability. Always always always when the day comes the Excel craziness over colors drives me to leave for another, sadly similar, position - they are surprised that the younger (cheaper) version of me can’t get the chart to work - let alone layer in that shade of vermilion.
-
Friday 9th October 2020 21:06 GMT Anonymous Coward
Re: Excel is Hell, but my IT support was downsized & laid off years ago!
This one post anonymous... I work with a team that at one point in the past (and may still for all I know) used colour formatting of cells as a way of storing information. Not conditional formatting, regular formatting. Red category A, green category B, that kind of thing. Apparently a lot of discussion went into the colours to use.
-
-
Sunday 11th October 2020 21:48 GMT MachDiamond
Not for large data sets
Excel bites for large data sets. We had problems using it to analyze telemetry data at a small aerospace firm that we'd get back from the rockets. I found something else that worked much better, but I'm blasted if I can remember the name of it. I've been out of that company for years now and good riddance.
If it's the only tool in your box, that's what you use. I'm the sort of person that looks for alternatives to the alternatives. I'm always working on unique projects that mainstream software companies have never considered except for one that was in the same position and wrote something custom and put it up for others to buy or use.
I also try to test my methods several ways to make sure my results are accurate. They® don't teach that in schools. One usually learns when they get their XXX handed to them by their supervisor that could just look at the numbers and know there was something wrong.
Biting the hand that feeds IT
