Evil third-party screens on smartphones are able to see all that you poke

Smartphone hackers can glean secrets by analysing touchscreen user interactions, according to new research. Boffins from Ben-Gurion University in Israel have shown it's possible to impersonate a user by tracking touch movements on smartphones with compromised third-party touchscreens, whether they're sending emails, conducting …

[Waseem Alkurdi]

Who else would have a fox’s cunning that brings these attack vectors coupled w/ spare time?

Who but the Boffins from Ben-Gurion University in Israel , home of the HDD activity LED blinking hacking?

Hats off to you and your work though. We appreciate the effort.

But there’s a problem with this technique: What if the touch event capture stream is out of sync with the actual input?

Let me explain. What if the “recorder” misses a tap or two? That’ll screw up the whole session/tape/whatever.

What if a user “cancels” a touch (by holding down on an object then swiping off bounds)?

The hypothetical recorder still registers a swipe but no equivalent takes place in software - screwing up all subsequent recording.

[Anonymous Coward]

Re: Who else would have a fox’s cunning... "...then swiping off bounds..."

"What if....then swiping off bounds..."

The creators of the malicious code could, conceivably, include that possibility in their table of gestures.

I know it's difficult to imagine, but it is possible.

[DaLo]

Re: Who else would have a fox’s cunning that brings these attack vectors coupled w/ spare time?

It can't "miss a touch or two". It would be built in to the screen's digitiser. Therefore if it missed a touch then it wouldn't send that data to the CPU in the first place.

It could also "read the screen" so that it knew what information was being displayed and there what app was in use and what apps were installed potentially making it 100% accurate.

[Anonymous Coward]

Clever

You can't touch that!

[theExecutive]

Re: Clever

McCoward?

[Rob D.]

Bladerunner - the adult cut

Using a series of questions and games, the researchers employed machine learning to determine stroke velocity, duration and stroke intervals on specially modified LG Nexus Androids.

[DuncanLarge]

What?

Screens run code now?

[phuzz]

Re: What?

It's easier and cheaper to have a chip in the screen which translates between what the computer/phone wants displayed, and the actual electrical signals, than it is to have the computer output raw display output (which would have to be changed for each type of screen). This goes for input devices (like the touchscreen in this case) as well.

[Peter Gathercole]

Re: What?

It would not at all surprise me if the touch screen has it's own ARM core, or at least a microcontroller or PIC.

All of these could run additional code that captures all sorts of information, but I have to ask, what does it do with the data after it's been collected?

I very much doubt that whatever the controller is, that it has access to the higher levels of function like the IP stack, or even access to the main memory of the phone or whatever device the touch screen is fitted to.

Chances are that there is a definite protocol that the screen and main processor use (it's probably based on I²C or something similar) to allow the OS to identify what is touched and when, and unless this is a lot more functional than it needs to be, passing out-of-band data is not something that is likely to be very easy. You would have to have some system component added to Android (the applications are abstracted from controlling the hardware) to read the data.

So even loading a dodgy app. that looks to see whether one of these hacked screens is fitted, is unlikely to be able to query the screen controller to get that information.

What I would like to see is what the special modifications were made to the demonstration phone to allow this demonstration. I'll bet they included a modified Android driver to talk to the screen. If that's the case, then I'll breathe easily regarding the screens I've replaced on several phones.

[Anonymous Coward]

Re: What?

I wouldn't be so sure that the code only runs inside a chip on the screen. Some devices have downloadable drivers, or download code that's inserted in the drivers. Then it is running at full kernel level permissions, and it would be easy for it to get the data it stole off your phone and to those who will use it against you.

Even if you knew for sure data the screen captured couldn't find its way to the outside world, there's an easy solution for that - give people a one year guarantee on the dodgy screen, and have it automatically "break" after it finds something especially juicy, like bank passwords for someone with a balance of $5M. They'll go get a free replacement, and the evildoers get their hands on the data inside the "bad" screen.

[Peter Gathercole]

Re: What? @DougS

Interesting thought. I don't know enough about Android device drivers to know whether they can accept microcode from the device that is being configured, but I think it is unlikely.

Doing a bit of research, it does indeed appear that the interface for most screen controllers is I²C, SPI or SSI, and these do not require, and does not allow code to be injected into the device driver. It may be possible that it would react to a request for the device parameters from the device driver, but that would be at the device drivers request.

On the subject of recovering the device, these replacement screens are provided at the lowest cost, and the chances of someone actually following up on a warranty after they've had the device for a year is very unlikely. Generally, phone repairers working at this low end just buy direct from China over Ebay, and are unlikely to return the faulty device unless there was a compelling reason, so they would have to be complicit (the cost of returning stuff ti China is much greater than sending it from China). I'm not saying it couldn't happen, but...

[Waseem Alkurdi]

Re: What?

It would not at all surprise me if the touch screen has it's own ARM core, or at least a microcontroller or PIC.

All of these could run additional code that captures all sorts of information, but I have to ask, what does it do with the data after it's been collected?

@Peter Gathercole

I loved your post ... Very logical inferences.

I have to ask this though: Is it necessarily true that the data is going to be sent *somewhere*? The data could be stored on the "spy IC" itself in some form of flash storage. Then, when the phone is retrieved again (assuming military/intelligence force/organized crime in this) after killing the owner, the flash storage is dumped to a file and read.

[DJO]

A gift to Apple

Now they can use this research to claim third party screens are a security risk so iDevices must have a genuine Apple part fitted by Apple when a repair is necessary.

Of course they could supply screens wholesale or write a utility to test a screen but that wouldn't fit the Apple philosophy.

[Anonymous Coward]

Re: A gift to Apple

How would 'testing a screen' insure that it doesn't have malicious code? Testing can only determine that it meets the software specs, not that it doesn't do 'extra' stuff or verify that the hardware specs (i.e. calibration etc.) are met.

I do agree that Apple went about it the wrong way, what it should have done is produced a warning that the screen isn't a genuine Apple part and may not function properly when you boot. You can click through that and ignore at your own risk, or complain to whoever replaced it / sold you the part if you were told it was a genuine Apple part.

[Zog_but_not_the_first]

So Poke...

... then Peek.

Thanks for that nostalgia hit.

[EveryTime]

Touch screens are generally I2C or SPI connected, and have a very limited protocol. The chips used on high resolution ones are specialized -- it would be a major engineering effort to create one that had introspection on what the touch input means, multiplied by the need to fit within tight power constraints.

There are exceptions. NVIDIA integrated a touch controller, 'DirectTouch' into later Tegra chips. They advertised it as reducing latency, but it was in part to justify the high SoC cost by moving functionality on-chip.