London Mayor election day bug forced staff to query vote DB by hand

The confirmation of Sadiq Khan as Mayor of London last month was delayed for several hours by a database application bug. The announcement of the election's winner was pushed back from May 6 to May 7 as a result of the programming cockup, which was buried in the IntElect systems used to scan and tabulate citizens' votes. …

[Paul 25]

WAT?!

How is this software not tested to within an inch of its life?

"Counting up the votes" seems like the most fundamental feature of the software, for that to not work reliably really does cast doubt on the abilities of the company in question.

[AMBxx]

Re: WAT?! - software

You're overcomplicating it. For software, read 'Excel Spreadsheet'. They're using Excel 2000 so hit the 65535 row limit.

[steogede]

Re: WAT?! - software

According to the BBC, you're not far of the mark. Not sure what the world is coming to when the BBC have more technical details than El Reg.

> Staff had to collate discrepancies in a spreadsheet, rather than automatically, when tallying final results, said chief executive Steve Gowers.

> A resulting investigation found that the underlying data, the actual numbers of votes in the database, was correct - but the counting software was not reporting it accurately.

> The decision was then taken to disregard the system's ability to collate results from each constituency - querying this data was instead done by programmers.

> "The manual part of the process meant that we took the output of those queries and put them into a spreadsheet to add them up," explained Mr Gowers.

http://www.bbc.co.uk/news/technology-36558446

[Anonymous Coward]

Re: WAT?! - software

Scenario 1: Votes are counted at one go, result is not Sadiq, tamperer changes database but find the voting software updates the vote totals but no the grand total.

Scenario 2: Voting software reports votes entered originally, not the count of DB records. Database has entries deleted to make Sqdiq win, voting count mismatch.

Scenario 3: Crap software that somehow fails even a basic count.

It makes no difference which of those it is, the count is void, the software and this the database are untrustable.

[diodesign]

Re: steogede

We here in San Francisco published the news hours and hours before the BBC did by listening to the Assembly meeting. At the time we couldn't reach DRS for comment and went with what we had for the UK early breakfast time.

It looks as though the BBC got more info out of them during the day, and really the extra info is that they used a spreadsheet to count up the totals. The main problem, that their software was broken and they had to manually query the database, was broken by us hours ago.

So, OK I'll add in the bit about the spreadsheet. We strive to get all the techie details into stories, but sometimes we have to call time and publish what we've got.

C.

[gv]

Re: WAT?!

I'm curious as to what development methodology they followed: was it agile, waterfall? Did they use Kanban? Were DevOps involved? What language(s) were used for the implementation? Did they outsource/offshore the code? What reviews were undertaken? Who reviewed the test scripts and results?

[AMBxx]

Re: WAT?!

Sounds like Agile. Keep using it at successive elections until it works.

[Terry 6]

Re: WAT?!

Never mind the technical questions.

Just what does " (Gowers insisted the system had been tested prior to deployment.) " actually mean.

It could be as little as something went in and something came out.

[robidy]

Re: WAT?!

I think you're right, it all to easy to say something works, it's a damn sight harder to run a sensible test plan to prove it works as described.

[Anonymous Coward]

Re: WAT?!

'It could be as little as something went in and something came out'

I've found tests like that:

input = new TestInput

result = new Result

class = Mock.Class()

mock.class.ValidateInput(input).Return(true)

mock.class.Save().Return(true)

mock.class.ProcessData(input).Return(result)

result = Class.FunctionToTest(TestInput)

Assert result.IsNotNull()

[Anonymous Coward]

Re: WAT?!

It means the election is null and void. A talley miscount is a clear sign of tampering, claiming the data into the database is correct is a claim they could not know without manual rechecking each entry. So it is a false claim.

Ergo the election count needs to be voided and the paper ballots checked and counted by hand. If there has been the possibility of tampering with those ballots then the election needs to be rerun completely.

You cannot have a situation where an election official asks a contractor who won. The election officials job is to determine who won, not accept someone elses word for it. Hence Sadiq did not win the vote, he won a contractors opinion as to the vote result.

[Primus Secundus Tertius]

Re: WAT?!

When I stood for election to my local council, I saw ballot papers marked with a tick instead of a cross.

Does the scanning of ballot papers cope with this?

[d3vy]

Re: WAT?!

No because people who cannot follow simple instructions should not be allowed to vote.

[Dan 55]

gv: It'll be a "commercial secret" even though all of this should be open source.

[Chewi]

Apart from all of the above, how hard can it be to add up a bunch of votes?

SELECT candidate_id, COUNT(*) FROM votes GROUP by candidate_id;

Okay, so voters got a second choice but even still.

[AndrueC]

SELECT winner FROM candidates;

Simples.

[kmac499]

I'd stick a DISTINCT in there somewhere. that should take care of fraud.

[Adam 1]

That query isn't very helpful

Winner

------

False

False

False

True

False

False

False

False

[Adam 1]

Just tried kmacs suggestion but got

Winner

------

False

True

Screw this. Going to stackoverflow to get a proper answer.

[Wilco]

If it sounds dodgy, it is dodgy

It should be a legal requirement that all software used to decide the outcome of elections should be open source. Determining the winner in elections run on a non First Past the Post basis (Single Transferrable Vote and various forms of PR) can require some relatively complicated algorithms. The implementation of these algorithms needs to be open to public scrutiny so that it is harder for errors or deliberate cheating to affect the outcome of the democratic process.

Human counted election are transparent because the counting is done in public by a large group of people.

Computer counted elections can only be equivalently transparent if the source code is open to all for inspection.

And there's no need for this code to be closed source. The vendors have to adjust their business models from "selling election software" to "providing election vote counting services"

[robidy]

Re: If it sounds dodgy, it is dodgy

Quite right, make it clear and open.

[David Pollard]

Re: If it sounds dodgy, it is dodgy

Computer counted elections can only be equivalently transparent if the source code is open to all for inspection.

Open source is vital. It is also necessary to have a permanent record of votes so that a recount is possible. Electronic storage doesn't allow for an audit using effectively unchangeable data in the way that paper and pencil does.

[SkippyBing]

Re: If it sounds dodgy, it is dodgy

'effectively unchangeable data in the way that paper and pencil does'

I've got a trick with an eraser that might blow your mind...

[Rich 11]

Re: If it sounds dodgy, it is dodgy

I've got a trick with an eraser that might blow your mind...

If it's the one I saw in Amsterdam in 1989, you're too late.

[cosmogoblin]

Re: If it sounds dodgy, it is dodgy

The vendors have to adjust their business models from "selling election software" to "providing election vote counting services"

They don't even have to do that, they just have to publish the source. They don't need to go the full GNU route of allowing anybody to use their code, and they can still sell it on a traditional, subscription, or per-election basis. It's not like anybody is going to steal their code - it's of no practical or business use to most people, and if the government used their software without paying, it would be pretty bleedin' obvious!

[veti]

Re: If it sounds dodgy, it is dodgy

While I agree that open source is important, that's not going to make it anything like "equivalently transparent". How many people are qualified to read and fully understand the source code?

And the compiler?

And the OS it runs on?

And the hardware the OS is installed on?

If you don't understand all of that, how can you be confident you really know what it's doing?

The only really, meaningfully "transparent" system is the one where people sort bits of paper into piles, and count them. In front of as many witnesses as can be bothered to turn up. Anything else is a huge loss in transparency.

[d3vy]

Re: UK elections.

F*&k off.

I do a postal vote, because I work, my wife works and we have two young kids. Going to a polling station would be an almighty ball ache.

Add to that I am a contractor and could be called to the other end of the country the night before the vote.

There are lots of people that simply won't vote if you take these options away, of they open the doors for EXTREMLY small scale fraud, really how much of a difference does one vote really make? If an election is close enough that one vote swings it I'd me more worried about miscounting than fake postal votes.

[d3vy]

Re: UK elections.

I would add that my ball ache becomes an impossible feet for many who simply cannot make the trip (pensioners etc) or who will not be in the area to vote.

[Anonymous Coward]

Re: UK elections.

Feet have balls, yes, but did you mean feat? :)

[x 7]

Re: UK elections

"ball ache becomes an impossible feet"

big feet treading on your own balls?

[d3vy]

Re: UK elections.

"What I'm saying is that postal and proxy votes should be the exception, not a matter of convenience. You f*&ker! :-)"

I said nothing different, just that the exceptional circumstances should be quite wide.

For example, you might be ok taking your kids but if I'm away my wife has two options.

Take my two kids (one autistic) and queue to vote.

Or don't vote.

As I am unexpectedly not in the area I can't vote at all..

This is why our default if postal votes, it's that or we dont bother.

Exceptional circumstances will vary from person to person and you cannot prescribe what those should be without excluding voters.

[Dan 55]

Re: UK elections.

However they haven't done everything to make a polling station voting "accessible". There's no law that says voting has to take place on a Thursday, it's just a tradition. Elections and referendums* could easily be held on a Saturday or a Sunday and it'd be much easier for people to turn out meaning postal voting would be used less. As there would be fewer postal votes they could be scrutinised more and maybe votes could be permitted if only a good reason can be given.

Until they do that it could be claimed they're not really interested in reducing the chances of postal vote fraud.

* Okay, referenda.

[MonkeyCee]

Re: UK elections.

Well proxies can have that issue, but that's surely a problem with whom you select as your proxy.

A postal vote (or equivalent online vote) IMHO is *better* than an in person vote. I'm OK with there even being a record of who I voted for, that I can check. I do understand why some people are ashamed of who they vote for, so need privacy, but it would be far easier to ballot stuff rather than forge postal votes.

Tower Hamlets is a basket case for electoral fraud, in the same way that Chicago is in the US. It's very much the exception, but much like 110% turnouts the fraud is blatent and obvious. Still bad, but most places are quite a lot better.

My main issue with the UK system is that I've only had two votes that mattered in my lifetime, the AV change, and the EU one coming up. FPTP means that unless I can rally 20k+ like minded people then my vote is without meaning. In NZ, at least one of my two votes always meant something, and the FPTP part was more contested.

[keithpeter]

pivot tables?

"However, the code that queried the database to pull out the totals was not working properly, leading to "small discrepancies" between the number of paper votes and the reported totals."

Dump it out and pop it into an Excel sheet and do a pivot table? Its only 2 million+ rows.

Seriously: I support the idea of requiring all software used in elections to be open source (or source code available). Similar moves in science have gained some support already...

https://opensource.com/life/14/6/respected-journal-makes-transition-open-science

[caffeine addict]

AV

Hang on - looking at that polling card, does London have an AV vote for London mayor? The same system that we decided was too expensive, or complicated, or whatever the hell it was for us to use in the rest of the UK?

[Anonymous Coward]

Re: AV

Technically the London Assembly uses the Additional Member System for elections ( https://en.wikipedia.org/wiki/Additional_Member_System ), which is of course far more complicated and expensive (considering the use of closed source software at who knows what price) than AV ever would have been.

It seems shocking, but often politicians lie in order to get the desired outcome from a national referendum.

[Nigel Whitfield.]

Re: AV

Technically, for the assembly, I think it's a modified d'Hondt system, which allows you to maintain the constituency link, but have a proportional result.

For the mayor, it's technically Supplementary Vote.

Meanwhile, elsewhere in the UK, we use Single Transferable Vote for NI Assembly, local elections in Scotland and NI, and European Parliament elections in NI.

The Additional Member System (that d'Hondt method) is also used for Scottish Parliament and Welsh Assembly. And Closed Party List for MEPs except in Northern Ireland.

And then FPTP for most other things.

No one ever pretended the UK made sense...

http://www.parliament.uk/about/how/elections-and-voting/voting-systems/

[energystar]

"The manual process was more transparent and allowed us to do more checking."

Here we have people saying that the King has no clothes. Too much time seeing the cards doesn't amount to having seen all the cards.

[Jamtea]

Assume that the 'software' in question was some official's nephew straight of college who said it would be 'easy' as he runs a Minecraft server.

Honestly, I question the abilities of this so called tech company, when a simple SQL query on pretty much any client would have totalled up the votes in a matter of seconds.

[Anonymous C0ward]

"It is enough that the people know there was an election. The people who cast the votes decide nothing. The people who count the votes decide everything."

-- Joseph Stalin