Data-mining technique outs authors of anonymous email Engineers and computer scientists say they have devised a novel method for identifying authors of anonymous emails that's reliable enough to be used in courts of law. In a series of papers published over the past few years, the researchers from Concordia University in Montreal have described what they say is the first ever data …
Specifically, it would be easy enough to get a program to add randomness to a lot of these features in order to properly anonymise the writing style of the person sending the email. Loose some spelling acurracy, drop in some, broken sentences. even just dropping the odd capital would help reduce the confidence of the system until it was inadmissable in court.
Prose style would be far more awkward, but maybe the system could then highlight distinctive phrases and advise that you may very well wish to rephrase them?
"When finely tuned, the technique identified the author about 80 percent of the time."
In other words, they think a 20% failure rate is "reliable enough to be used in courts of law"?
Well, in combination with other evidence it might be, I suppose. But given the "believe anything the computer says" attitude of some people I doubt it.
Given that 80% of spam is written at the Lagos School for e-Business, and probably all cut, pasted, and mutated from one original "Hi I've inherited a squazillion of dollars, do you want for a fee to have a fraction of that?" writer. So strictly speaking, correctly identifying the original hand behind all that is not that hard.
Note that the algorithm is not such that it determines the person, it is such that it determines whether 2 mails probably come from the same person; so it relies on the author kindly identifying him/herself in one of them.
Would the police in the UK do anything with it ?
It took me over a week to get a respomse after reporting a crime under the misuse of computers act and the original response was that "if i had lost nothing then it was not a crime" ? I dont know why i bothered supplying them with with all the information they required to trace the offender.
It makes me wonder if this is a nice safe way to make money as the police dont seem to give a crap.
The majority of coppers are clueless when it comes to anything slightly technical so i cant see this being used to stop all the "Viagra, penis enlargement and SEO" emails i find in my inbox. If they are outside the UK what exactly can the police do anyway ?
And that's in cases where the authors were not trying to hide their identity or impersonate someone else. No way could that be "used in courts of law", particularly since that reference to "finely tuned" presumably means that a corrupt police force or expert witness could finely tune it to identify the person they want to incriminate.
20% failure rate is not much better than heresay. Which lawcourts are going to accept that as evidence? Just wanting to know so I can avoid those territories in the future...
I bet the hit rate drops significantly if you have always known this analysis will be applied to your writings one day and are subsequently paranoid about consistent phrasing and sentence structure.
80% is good enough for a court of law? Out of 200k emails, they misidentified 40,000... With a sample set of fewer than 200 people. I don't think that's enough for probable cause to have a warrant issued in the US, nor sufficient for presentation as evidence in a criminal case. Presumably with a larger pool of people, the accuracy goes down further - I can think of very few situations in which the software in its present state would be useful.
And there I was, thinking that "malicious email" was about the payload that would take over your computer when clicked upon.
So tell me, what is the minimum amount the algorithm needs? Will this be enough:?
"Hi! How are you?
I send you this file in order to have your advice!
See you later. Thanks"
On a more serious note, this algorith has, given a sufficiently large corpus and after a sufficient amount of fine-tuning (with a-priori knowledge? how else would they tune?), a 80% chance to pick the right author from a list of suspects. That's not quite lifting the veil of general anonymity. Wonder how well reference humans would do, sufficiently tuned through being familiar with the writing style of (all of) the suspects. Did they perchance do a comparative study?
Also, what is it that makes this /suitable for court use/? The 80%? Is that it? Is the enron corpus the only one that they used to test this with? Then I wouldn't be so sure they'd get 80% elsewhere too. In theory maybe, but that doesn't automatically scale to the courts. Go find a few other bodies of emails to try this on, and we'll see what happens then.
"When finely tuned, the technique identified the author about 80 percent of the time."
And that is "reliable enough to be used in courts of law" is it?
Beyond reasonable doubt? I think not.
And what does "finely tuned" mean exactly? Might it have something to do with knowing the answer in advance? What is its accuracy when not finely tuned?
"...reliable enough to be used in courts of law"
"When finely tuned, the technique identified the author about 80 percent of the time"
That is really going to impress a court of law, or has the situation changed where the MD5 sum was considered not reliable enough for electronic evidence in spite of being much, much better than this:
http://www.schneier.com/blog/archives/2005/08/the_md5_defense.html
Unless of course you are accused of a crime that attracts witch-hunt like emotions?
Only automating it is actually original. The different components of write print as a method in criminology have been around for a considerable amount of time. A lot of criminals have been caught by their writing style. Some by luck when someone recognizes it (Unabomber). Some through a concerted effort and analysis.
However, prior to this these were the domain of proper police and used only for very serious cases where paying someone qualified to do a lexical analysis was worth it. Even then, it was classed as circumstantial evidence and used only along with other data.
The difference here is that this development puts this technique into the hands of people some of which are neither qualified to use it nor can understand the consequences of 80% maximum confidence (on a rather large statistical sample). I may be overly pessimistic here, but the wya this is going "tools" to for use this technique with an automated feed of the company Exchange mail server are at most 2-3 years away.
So the next "anonymous" personnel survey you will be filling may actually be way less anonymous than you think...
If it is only correct 80% of the time, I'd say that it doesn't go as far as "beyond reasonable doubt".
Besides, I suspect it wouldn't be difficult to imitate somebody else's style, given a few examples of their emails, which leads to the question : how do you identify somebody if you only have one anonymous email?
You'd have to have a good idea of where to start looking in the first place : no samples = no match.
I'd imagine that it would be pretty easy to protect yourself from this technique by using a web translation service to translate your email to another language, and then back to the original. That would probably give enough of a difference in writing style to screw up the results while keeping a relatively legible email that stays true to the original.
I imagine it would be fairly easy to protect yourself against this technique using a translation service for websites to translate your e-mail to another language, then back to the original. That would probably give enough of a difference in writing style screw results while maintaining a relatively readable e-mail that remains faithful to the original.
Google translate English to French and then back again
With a 200,000 sample and a limited, known user populationof 158 the best they got was 20% of them wrong, yet based on this high error rate under optimal conditions, its good enough for court?! I don't think so!
Of course peoples personal style and content in a corporate environment is going to use a common, person specific format, but an 80% hit rate is nothing to rave about. Come back when you are in the high nineties for a non-MEH response
What's to stop somebody deliberately changing style? We could have a digital copy-and-paste return to those old blackmail notes that used cut-out newspaper headlines.
The anonymous writer could just copy a "suitable" phrase from one document and one from another and so on. Would that work?
"Engineers and computer scientists say they have devised a novel method for identifying authors of anonymous emails that's reliable enough to be used in courts of law."
Which court of law are we talking about here? IANAL, but have some experience of legal processes. In the UK, to secure a criminal conviction the burden of proof is "beyond reasonable doubt". I do not believe that this technique meets that requirement: "When finely tuned, the technique identified the author about 80 percent of the time" 80% correct = one in five incorrect. I doubt, reasonably enough.
I also doubt whether it would have much bearing on a civil case where the balance of probabilities is the test, unless backed up by other evidence.
When they stated "identified the author 80 percent of the time" did they mean:
(1) Given 100 emails it identified authors for 80 of them and they were all correct - the remaining 20 where marked as "unsure"; or
(2) Given 100 emails it identified authors for 100 of them, but upon examination only 80 of them were correct.
Option 1 is a decent piece of software detective work. Option 2 is a lousy negative match rate.
And this is supposed to be good enough for use in court? Holy hell...
With a false positive rate of 20% on such a small sample it's next to useless for picking people out of the general population, surely? All you could hope to get is "this guy we already suspect writes in a similar style to the release", which has got to qualify for pretty weak circumstantial evidence at best.
80 percent is reliable enough for a court of law? 20 percent constitutes reasonable doubt in my mind.
A thesaurus and a list of common spelling and grammar mistakes tied into a random character / word replacement / transposition script ... I'm fairly sure someone has thought of this already.
Your move.
We start with "... that's reliable enough to be used in courts of law."
And end with "When finely tuned, the technique identified the author about 80 percent of the time." Out of a very limited set of 158 suspects, but with at least 200,000 emails to chew on.
This has no teeth.
I hope the 'engineers and computer scientists' behind this research involved linguists at an early stage. There are enough debates among experts in that field as to the authorship of various literary works to call this whole idea into question. For example, how much of the work attributed to Shakespeare was really penned by him? I suspect it is far to easy mimic the writing style of another person to make this stand up in court.
Whilst 80% for a single document wouldn't be good enough, suppose you have 4? All identified to the same author. That gives you a pretty low chance of the suspect not being the author. Building a case to a "beyond reasonable doubt" level involves multiple levels of evidence. In a harrassment case, your likely to have several emails (otherwise it wouldn't be harassment). That's the point of building a case - if they were all easy, police wouldn't have much to do. It's the same process we use in the brain to decide if something is true or not - we build evidence until we hit above our "truth threshold" and decide it's true.
Under criminal law the onus is for the prosecution to prove "beyond reasonable doubt" - I agree that is not met by this 80% accuracy.
However a civil case it is only "on the balance of probabilities" so anything over 50% is technically acceptable.
So good enough for a libel or copyright case .....
Oh shit.
You know that hocus-pocus personallity test of symmentrical ink-blots that all look like female genitalia, for which you need to answer what this most reminds you of so that you appear as 'normal' as possible to the testers, while trying not to snigger / not get a boner / not spoil the test-paper?
Well, the Rorschach Test is still being widely used in the US & Canada. Small wonder then that this email data-mining technique is also treated as credible evidence.
Based on my previous (named) postings, I wonder if this process would be able to divine my identity from this anonymous post? Ha!
I would see this being used as a tool to reduce the number of possibilities for an author
in the test they did where it got 80% right out of 200,000 it doesnt sound too useful but if you apply that figure to trying to identify the author of 5 emails amongst that 200,000 and the system comes back with the same name 4 times and a "no match" or different name for the fifth I know where I would concentrate other (human) resources
process it and then toss it out in whatever language style you wanted.
Some were really, really different. It's a bit like a word processor with a selectable grammar option - business, casual, etc.
Bet that would bugger up even that great university, Concordia. Great for untouched e-mails but undoubtedly beatable if need be.
Businesslike:
I'm concerned that this study doesn't seem to address the fact that people tend to change style depending on the target audience.
vs casual:
Doubtful. We speak differently to different people.
vs. Anonymous:
Bollocks. Do these retards think I'd post words like this if they who I was? *
* I know, not too hard to discover, but Anon posters often don't think that far ahead.
It can only tell you which of the suspects has a style most like the author of the email. Or put another way, if the author is not known, this method won't unmask them.
Essentially it could be argued that there is no way to prove that there isn't another person with a similar educational background and a similar writing style.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
