Users complain of missing data in UK wills search service

Users have complained of missing data and trouble logging in after the UK government updated its old probate search service. Run by HM Courts & Tribunals Service (HMCTS), the probate and will search service is a handy tool for hunting down probate records for documents and wills in England and Wales. It can be useful for …

[Screwed]

Why not index all names?

Have been looking for a will of someone who died in October 2020. This is a member of partner's family, and she is simply interested - not expecting to have been left anything, but would like to be sure. Despite being the current live service, it is emblazoned "beta This is a new service – your feedback will help us to improve it."

His name wasn't there before the "upgrade", and after the service did eventually come back, with the unintuitive interface you mentioned, he still isn't there.

Worse, it currently shows no-one with that surname was granted probate in 2021 - though there were several (from memory, around 7) before the upgrade.

From my point of view, it seems crazy that I can't see his name at all. We know some wills are contested. And others are late for many reasons. But the system won't even list the name if there is an outstanding caveat or for some other reason has not been granted probate.

There seems to be no obvious reason that the name isn't added to the database as soon as the probate registry first find out about the will. Indeed, why doesn't the probate registry pick up deaths from the registrars and automatically add them?

[Yes Me]

Re: Why not index all names?

It's manifestly incomplete. My grandmother is there, my grandfather who died a few months before at the same address is not there. These were both in 1968 (i.e. paper records) so there's no excuse for gaps.

One bit that seems to work well is the bit where you pay £1.50 a shot. I bet they tested that bit.

[Skoorb]

Re: Why not index all names?

It's a bit poor, but if you can't find it online, you can still request it manually using the form at https://www.gov.uk/government/publications/find-a-will-or-probate-document-form-pa1s. It still costs the same (£1.50), but takes four weeks. Even with a four week wait it's preferable to just not getting it at through the online system or waiting for them to fix the online system.

[Anonymous Coward]

Re: Why not index all names?

It's not four weeks. If you ring up after four weeks to chase something, they tell you to call back after eight weeks. After that you speak and send emails to different people and hope that one of them might be able to do something about it. Chances are you'll have to explain everything all over again to each person you talk to. If they do actually have a CRM, it's pretty useless.

[Chris G]

It sounds like like old school engineers, they have put a tent up over the servers, have closed the entrance and are sitting inside drinking tea like the telephone engineers of old.

[Ben Tasker]

By sheer coincidence, I was using this service earlier.

It's not just the search they've made a hash of. Actually creating an account on there was a pain because they've made a mess of enforcing password requirements - https://twitter.com/bentasker/status/1495717477208236040

When you enter a password that doesn't meet requirements, the interface will

- Display your password in the clear

- Tell you it must have at least one lowercase letter, at least one uppercase letter, one special character and at least one digit.

At which point you (and anyone looking over your shoulder) will read your password and think "but it meets that?". They've done that old thing of not including all special characters, and not telling you which they don't accept.

The ones I could find that aren't accepted are =\+.

Eventually, I had a password accepted because it had a ? in it.

Until I saw this story, I'd assumed it was some cruddy old implementation that noone had bothered to drag into the modern world...

[Claverhouse]

Eventually, I had a password accepted because it had a ? in it.

Shudders.

I am not keen on using that character on the web...

[Kubla Cant]

Special characters

Where did the stupid term "special characters" come from? Special to whom?

Assuming that "special" means any character that isn't alphanumeric and isn't a control character, why does every bird-brain that implements this restriction have a different concept of this set?

Is any Unicode character legal, so I can have a password that's mostly emojis? I've never done this because I have enough password trouble to be going on with, but it's an amusing thought.

[Screwed]

Re: Special characters

How I hark back to the days when I could set . (a single fullstop) as my password. (This was when there were literally no external connections. Nonetheless, a dumbcluckstupid thing to do.)

It was so long ago that many didn't realise you could use any "special" characters in passwords - so unlikely to be guessed. Most were pure alphanumeric.

[John Brown (no body)]

Re: Special characters

I wonder if any password crackers actually start by testing for single character passwords? Or is it such a "dumbcluckstupid thing to do" that no one ever bothers to check? On the other hand, few system would allow that these days with their super complex requirements.

[TRT]

Re: Special characters

I've often wondered why sites don't implement some type of form input meta-data that informs e.g. Apple keychain, what are the minimum requirements for a password.

All too often I've found that Apple's rather handy random password generation tool falls foul of password restrictions on a site which means you have to fall back to old school methods of putting it in, which usually ends up as something like catchurchball&N33DC4P52!!!

[I am the liquor]

Re: Special characters

The real answer is for the sites not to enforce composition requirements, which has been the official CESG/NCSC guidance for some years. Specify a minimum length - nothing more in terms of composition - and then check the password against a dictionary of known weak or compromised passwords, is what they should be doing.

[TRT]

Re: Special characters

That's what they SHOULD be doing of course... but many don't allow "special" characters for some reason, and often you don't know that until it gets sent to a backend process. Sometimes of course the password validity is checked before being submitted. There are some reasons why you wouldn't want to submit some unusual characters, but then one should really be encrypting the password before transmission anyway, just in case! There are also maximum password lengths on many sites.

Anyway, besides all that, password suggestions generated by autofill suggestion tend to be restricted to not include special characters or only the dash and the rest is made up with random upper and lower case letters and numbers. They also limit themselves to e.g. 16 characters only. I suspect this is an attempt to maximise compatibility with websites, but it does restrict the range of passwords again - a sort of composition requirement by default.

If the PATTERN attribute of the password input, for example, was used to indicate acceptable values, then such auto-filled suggestions could better match the coder's requirement at either the back end or client-side. It would be totally possible to have a regex which simply hints to the suggestions composer which special characters are allowed and length, say, between 8 and 64 characters long:

(?=(.*[a-z]){0,})(?=(.*[A-Z]){0,})(?=(.*[0-9]){0,})(?=(.*[@%+\/$^?:,(){}[]~´-_.]){0,}).{8,64}

(escaping removed for clarity)

That is in this example any 8 to 64 character string made up of any number of lowercase, uppercase, numeric or special characters from the list. It would be great of course if this was somehow given a "shorthand" value such as "ISO_27001"

This would allow these random generators to become even more random with some confidence. It doesn't stop the site from additionally parsing the supplied input against a cracking dictionary or compromised list.

[Phil O'Sophical]

Re: Special characters

That kind of algorithm tends to fall over when presented with non-8-bit characters, especially if the front- & back-ends aren't using the same settings.

[TRT]

Re: Special characters

Multibyte regex is a thing. Though the W3C spec for regex in the PATTERN attribute doesn't specify if it should be mb aware or not. I think. Unless it's in a definition somewhere.

[hoola]

Re: Special characters

Even better was some web management portal (I think for PlusNet) who changed the password requirements so enforce length and complexity including special characters. Now as an existing customer I did not know this.

I came to login one day to do something and could not get in, WTF....

Check the password, it is correct, retry, still "incorrect username or password"

Check router is logged in and when it last dropped and reconnected, all good.

I did not do "reset my password" because I knew it was correct. Phone them and get through to someone who tells me the password needs resetting because I have forgotten it.

Okay, reset set the password and put it back to what it was to humour them.

Still cannot login.

Continue phone call and get through to someone who is technical.

In the end they ask for the password so okay, I can reset it again, it is random, no issue.

He then says, oh, you cannot use that password because of this character.

So this takes us full circle, they changed the requirements and had a list of excluded characters that meant you could not log in, however you could change the password and use one of the excluded characters.

[Terje]

Re: Special characters

Or simply allow Unicode straight off and doing it in reverse and specify prohibited characters (this should be done for logins as well) that way you are not arbitrarily limiting names and passwords to what the rather limited English alphabet.

[John Brown (no body)]

Re: Special characters

"check the password against a dictionary of known weak or compromised passwords, is what they should be doing."

I can understand the check for weak passwords, by why check against known compromised ones? Unless, by that, you also mean check the username and password against known compromised ones. After all, what does it matter if some person Jim46 used P455w0rd36 on some site and I then use the same P455w0rd36 with my username John.Brown on a totally different site?

[Len]

Re: Special characters

Because dictionary attacks don’t only use a dictionary of existing words, they often also contain lists of common or known (because they were uncovered in earlier breaches) passwords. They probably even try those first before they try a normal dictionary.

[Anonymous Coward]

Re: Special characters

Or, put another way... if you think that 'P455w0rd36 ' is a cunningly clever, super-secret password then it's odds on that someone else will as well.

[John Brown (no body)]

Re: Special characters

To be honest, that's my point. Unless the password is truly random, non-human-rememberable, then it's highly likely someone else has also used it somewhere, and possibly had it "discovered" by h4xors and added to "The List". I suppose it's really a case of assessing how important a password is in each particular circumstance. The more important sites will hopefully offer 2FA ,where they will know exactly who you are anyway, so them having a phone number for 2FA is no big deal, eg a bank will have your number anyway, or less secure but still better, limited tries to log in. It's hard to run a dictionary attack if you only get 3 tries before either a cooling off period or a lock-out.

[I am the liquor]

Re: Special characters

Well good news John, P455w0rd36 is in fact not in any password breaches known to haveibeenpwned.com.

Too bad it's been burned now!

But yes, as Len described, once a password has appeared in a breach of plaintext passwords (probably pinched from some site that stored them in the clear or used weak hashing), then it's in every password-cracker's dictionary. If they lift a database of password hashes from a site where you used one of those passwords, then they will decrypt it via a dictionary attack, even if an expensive hash function was used in that case.

Apparently haveibeenpwned has a dictionary of "hundreds of millions" of exposed passwords. So in cracking terms, any password that's in that list, no matter how long or complex, is reduced to about the level of a 5-character alphabetic password (380 million combinations).

[ShadowSystems]

Re: Special characters

You mentioned using Emoji in passwords, possibly tongue in cheek, but there was an article about that very abomination. I wish I could remember where I read it so I could link to it as proof.

The story was gushing about how *more* secure it could be as the folks using the method "can't be spoofed by foreign language (look alike characters) attacks".

Because nobody would ever fail to notice the difference between the smiley face with the left eye as a wink & the right as a butterfly from the smiley face with the eyes reversed; or the slightly different shade of blue race car; or the rocket ship blasting off but tilted +/- $XX degrees up/down from the one they used; etc etc etc.

I wanted to reach through the site & slap the stupid out of the author, which means it'll get adopted as SOP in 3, 2, 1...

*Sigh*

[Doctor Syntax]

Re: Special characters

"any character that isn't alphanumeric and isn't a control character,"

AFAICR even control characters were allowed in old-style Unix passwords - except return, of course.

[Hans Neeson-Bumpsadese]

Re: Special characters

AFAICR even control characters were allowed in old-style Unix passwords

You speak the truth. I distinctly remember using Unix systems back in the early 90s where the password included <ctrl>character sequence (partly because it could provide a greater degree of security, but mostly because we'd learned it was a trick you could do and thought it was cool to use).

[Doctor Syntax]

Re: Special characters

Password: HHH^H^H^H

[Anonymous Coward]

Re: Special characters

Blame IBM. They introduced a set of "special characters" for punched cards with the EBCDIC format in 1964.

https://upload.wikimedia.org/wikipedia/commons/thumb/4/4c/Blue-punch-card-front-horiz.png/1920px-Blue-punch-card-front-horiz.png

[John Brown (no body)]

Re: Special characters

"a password that's mostly emojis?"

I'll have to try that sometime!!!

(Although only on a new sign up I don't care about; I doubt I'll remember the password for more than a few seconds since I tend not to "read" emojiis. We moved on from hieroglyphs some centuries ago!)

[Anonymous Coward]

Re: Special characters

Some of the Disney characters are special to me - mostly for nostalgic reasons

[Anonymous Coward]

Re: Special characters

> Some of the Disney characters are special to me - mostly for nostalgic reasons

The default Disney password is '©©©©©©©©©©©©®'

[Hans Neeson-Bumpsadese]

There's at least one website which I use which imposes such complex requirements on passwords that I've resigned myself to not remembering it, especially as I don't use the site very frequently...my standard operating procedure is to go through the 'forgotten password' cycle, set it to something suitable contrived and get let into the site

[I am the liquor]

At which point you start to think they might as well get rid of the password completely, and make the password reset process (probably an emailed token) the logon process.

[Doctor Syntax]

These days I use KeePassX and its random password generator.

[Ben Tasker]

Me too. It was a KeePassX generated password that fell foul of the stupid requirements earlier.

It's generator lets you exclude characters (or only include specific ones), which is great - but only when the site tells you what they'll accept.

[TRT]

See my previous comment on the PATTERN attribute, REGEX and hinting for these password generators.

[Anonymous Coward]

The best one is when the site silently truncates your password manager-generated automatically-typed password when you set it because it believes it's "too long", so when you go to log in, you get told you(r password manager) entered the password wrong. *facepalm*

[Doctor Syntax]

"some cruddy old implementation that noone had bothered to drag into the modern world"

As you've discovered it has been dragged into the modern world. Modern world means broken.

[JimboSmith]

An ex colleague of mine worked for a firm who had some custom in house software. This was protected with login and password which had to be changed every 45 days or something like that. My mate thought it would be unusual and thus unguessable to include a swear word or two in his password. He tried a test password out and it was rejected according to the error message for not having the requisite number of characters, special characters and uppercase lowercase. He was puzzled as his obviously did and so he tried with another combination. He discovered that most but not all of the rude words he tried were banned. However it didn’t tell him that, it just went on about not meeting the requirements which he had. He asked IT support who were equally baffled and got on to the dev team. It later transpired the developer (who had since retired) was a bit of a Mary Whitehouse fan and objected to anything vaguely rude. Therefore there was a long list of words that you couldn’t use as a password.

My mate argued that this was silly and somebody from HR said it would be against HR policy for use of those words in an email or in the office, this on a computer too. Therefore they should be banned, he counter argued that nobody else was supposed to know your password. It should be encrypted on the system and therefore this shouldn’t be an issue should it? He lost that argument and had to use words that weren’t on the list. He bought a copy of the encyclopaedia of unusual sex and used some of the more obscure words (with far ruder meanings) from that which the developer obviously hadn’t heard of.

[Jonathan Richards 1]

Three syllables

In a large (for the time) rollout of desktop PCs for a secure system, we had a password enforcement regime which made users choose from pronounceable three-syllable 9-character "words". For first use, the initial password was communicated to the user (solely) in a sealed envelope.

We had one chap who came to us saying that he could not use his password as it was "racist and obscene". "Fantastic!", we said, "You'll not have any problems remembering it, and nobody else will know!" but he insisted on telling us what it was, in order to demonstrate its total unsuitability.

It is now, by my calculation, twenty-six years since this happened, and the total suitability is demonstrated by the fact that I recall his password to this day: "mad-fuk-wog". We changed it for him, of course, because by then it was compromised.

I apologize now if anyone is offended by the telling of this yarn.

[Anonymous Coward]

Re: Three syllables

Sounds very familiar. But a lot more recent than that.

[Mark #255]

of rude words...

I vividly remember listening to the Mary Whitehouse Experience, doing "The Swearing Experience", and Steve Punt saying,

Now, I can say the word felch on the radio, because it's on a swearing frequency so high that Mary Whitehouse can't hear it. I can't tell you what it means, but if any listeners are aware of its meaning, can I recommend you buy a bottle of mouthwash...

[JimboSmith]

Re: of rude words...

You have partially mentioned one of my mates passwords. I think he told me his last one at that firm was 69FelchingGimpGag96%% I know he doesn’t reuse them so it’s not an issue to post here.

[Cuddles]

Could be worse. My favourites are the ones that somehow manage to have separate requirements for password creation and password entry. It usually seems to be an issue with length, which isn't checked on creation but then the password field for login only accepts a limited length. You can have similar fun with plus-aliases in email addresses when sites check for a "valid" email address by looking for a @ and . while it's being typed, but only do so when logging in and not during creation.

[Anonymous Coward]

Probate

Having had the pleasure of dealing with probate services last year, it's just turned into a wreck. It took six months to get a copy of a document.

Par for the course in the UK - there isn't anything that can't be made worse by more centralisation and cutting the budget.

[adam 40]

Poorbate

This seems counterintuitive though.

If the Govt has felched up probate so badly, then the granting of probate will be delayed.

And therefore the settling of the estate, and therefore the paying of death duties/inheritance tax.

So the Govt is hoisting itself by its own petard (again!).

[Just an old bloke]

Just taken this for a test drive. All of the above is true, password in the clear for failed pw creation, password requires special characters which is OK providing the special character is a ?.

The interface and logic is straight out of 2001, the least intuitive system, most user unfriendly thing I've seen in many a year.

[I am the liquor]

It's great that people can still make money building stuff like that. Should make a lot of us here feel more secure in our jobs!

[X5-332960073452]

Crapita?

[TimMaher]

Re: Crapita

ICL (now Fujitsu) surely?

[Jonathan Richards 1]

> The interface and logic is straight out of 2001

"Open the pod-bay doors, HAL".

[Anonymous Coward]

> The interface and logic is straight out of 2001, the least intuitive system, most user unfriendly thing I've seen in many a year.

Welcome to Cabinet Office guidelines which imposed a mandatory look and feel across all gov.uk websites.

The inability to search on anything but the first 3 letters is probably down to Agile - MVP innit.

[Anonymous Coward]

In fairness to the .gov.uk interface design, it was at least designed to be accessible. It's boring as hell, but at least it's easy to read ...

[GrumpBit]

Has someone re-thunk their Testimonial

I wonder if the Testimonial on the front of IM's Courts and Tribunals service is still valid ;)

https://www.ironmountain.com/uk/resources/case-studies/h/hm-courts-and-tribunals-service

[Doctor Syntax]

Re: Has someone re-thunk their Testimonial

The testimonial's from a manager - possibly one of those who signed off acceptance tests (there were UATs & they were signed off weren't there?). He's not a user.

[GreyWolf]

Can you say outsourcing?

Perhaps written by new graduates from a non-Western university where the teaching materials have not been updated in two decades.

Plus: I have to mention my personal hobby horse. Usability.

Too many developers think usability is a matter of taste and style, which only they can possibly comprehend.

Nope; usability can be measured (time to complete a task; number of errors/restarts over a series of tasks).

[Doctor Syntax]

Re: Can you say outsourcing?

Add possibility of completing task.

I've often thought that testing should be carried out by a group of 3. One is a developer; one is a tester, a user familiar enough with the domain the application deals with but not with the application itself; and an invigilator. The tester is only allowed to ask - and the developer to answer - questions of the form "Where does it tell me how to do X?". Nominally the invigilator's role is to enforce this rule; it's actually to stop the other two coming to blows. The actual purpose of the procedure isn't to test anything, it's to teach the developer how to design a user interface.

[Anonymous Coward]

Re: Can you say outsourcing?

The correct people to do usability testing are either the end-users, or a reasonable surrogate (no, developers aren't this!). Ideally, you want around 6 (if you recruit 1, you might accidentally have found an idiot, and going beyond 5-6, you are unlikely to find additional issues).

Jakob Nielsen's company has lots on it ...

[Trollslayer]

Ah, the government I have known...

Let's just say this isn't a surprise.

[Doctor Syntax]

"testing was performed after each development sprint."

Of course we tested it. Was it supposed to pass the tests?

It was being complained about nearly 3 weeks ago on soc.genealogy.britain. Obviously not improved. Equally obviously no provision to roll back in event of failure.

[Anonymous Coward]

It's the British way...

It's the British way, isn't it? A bit like the HM Land Registry that only covers about 80% of all land ownership so can never be used conclusively.

I think there should be a version of Murphy's Law for how things are done here (Colin's Law? Nigel's Law?) that goes something like: "Anything that can be designed will be designed overly complex yet wholly unsatisfactory for the greatest number of use cases, for purely historical reasons".

[TRT]

Re: It's the British way...

I think that's called Sod Murphy's Law.

[Anonymous Coward]

Taking back control

We've heard of that.

[Jim Whitaker]

Email addresses

Currently the service is failing to recognise my email address as valid. If the rest of the process is just as competent, it is a good thing that my needs are neither urgent nor process threatening.

[AndrooB]

This is not the only problem with the service but just a symptom of their whole IT capability and lack of project ownership and control. I lost my mother last year and they managed to loose hundreds of applications at the same time as this development was on-going.

My application was via a solicitor and after four months of running around in circles I finally got the service to admit that chunks of the day to day processing is done on spread sheets and 'cut and paste' can go wrong.

Due to their data protection and legal rules my complaint to them was dealt with a curt 'bugger off' email and my solicitor has never answered my questions over why it went wrong other then discounting my bill. They will not raise the matter any further and answered my formal complaint with the discount leaving me the option of taking them to the Law Society for incompetence ot nothing. After this length of time battling nearly every day I'm so worn down I just cannot cope with it and giving up.

It's a classic 'old boys' club, answerable to no one and so disconnected from the real world pain the process causes that I am ashamed to have worked with Iron Mountain and in I.T.

[Anonymous Coward]

My private healthcare portal

Password validation is some copied and pasted off-site jquery crap that managed to fail on three different browsers. Guess which browser it worked with... yes, the modern-day IE6 by Google. That piece of JavaScript should be about 10K at most and work in every browser over the past two decades.

Similarly it's impossible to change the bank account numher even with Chrome. The account number is recognised as valid but the submit button is not there because it was previously hidden when the number was being typed in (and therefore not valid) but the bit to make it reappear again fails.

I just can't with the state of modern IT.

Rant over.

[Anonymous Coward]

Tried it yesterday, then today. Changed browsers but still problems:

1. Feedback option for 'beta version' (at top of screen) isn't working, just goes to a completely blank screen.

2. Can't see the top of the page in the results, not the first entries; can't see the year at the top of the page that lets you know you ARE at the top- this is no matter how much you zoom out (60% is the limit, btw)

3. After filling out the form, and then viewing the order, the dates have been changed. Both the date of probate and the date of death. Why ask if the software changes the date??

Not confident that I will get the right will back if the dates change so have given up. I live outside the UK so other ordering options aren't really practical.

This is unbelievably bad. It worked very well before, now it's not usable at all.