Sad SACK: Linux PCs, servers, gadgets may be crashed by 'Ping of Death' network packets It is possible to crash vulnerable network-facing Linux servers, PCs, and gadgets, or slow down their network connections, by sending them a series of maliciously crafted packets. It is also possible to hamper vulnerable FreeBSD machines with the same attack. Given that Linux powers an incredible amount of stuff these days, …
I don't think the /proc/sys/net/ipv4/sack workaround is going to work on all or even most linux systems. On my Slackware 14.1 system, the corresponding item might be /proc/sys/net/ipv4/tcp_sack ... or not ...
BTW, /proc is an internal pseudo file system, not a directory on the disk so modifications to /proc and its kids won't hang around when the system is rebooted. The Red Hat link has a more persistent patch (for systemd init).
I would guess that most ipv4 systems are NATed and that the device that is going to crash if they are assaulted by a zillion tiny message packets is the network router, not the individual PCs. Lord only knows what the router internal software looks like although I'd guess that it started off as a Unix of some sort.
But maybe I'm wrong.
Perhaps can use sysctl as well
root@yehat:~# sysctl net.ipv4.tcp_sack
net.ipv4.tcp_sack = 0
root@yehat:~# cat /proc/sys/net/ipv4/tcp_sack
0
(having just set it manually via sysctl as well as in /etc/sysctl.conf I did not touch /proc directly)
System is Devuan, so no systemd.
(Debian user since 1998, don't like systemd)
Tried it on Ubuntu 16.04 LTS too same result.
Much thanks!
Happy Devuan user myself, done that on all the systems on the offchance there isn't a patch yet (although if I wait till some day later next month an hour or so, there'll be one I'm sure)
Oh look, a kernel update as well..
"A reboot is required to replace the running dbus-daemon. Please reboot the system when convenient."
This is just plain WRONG. As Microsoft has taught us, the reboot MUST be forced at the most inconap'9rai9t4;o NO CARRIER
1) The ACK thing is an end-to-end mechanism - devices between just pass packet around, they don't mess with ACKs. So I think NAT would save you if and only if it doesn't allow a machine to be contacted from outside. Any NAT that publishes machine outside won't stop this attack
2) Not a few router/firewall that do NAT are themselves built on Linux/BSD - bring 'em down and everything behind is cut out.
3) Many "IoT" devices could be built on Linux too - and directly connected.
So, good it's not a RCE, still if you can bring down a service, it may still mean lost business (see Netflix...) or worse.
"modifications to /proc and its kids won't hang around when the system is rebooted."
Just add a line to an init script to make the mod.
"I would guess that ... the device that is going to crash... is the network route"
That's the worry. Are proprietary routers going to get firmware updates?
1) Routers don't ACK anything but packets addressed directly to them. ACK is a mechanism used by the receiving end to tell the sender which data it received (or didn't). TCP design is end-to-end - routers must only forward packets to their destination. If routers had to ACK, a) each router in the path should send an ACK, so the sender would get n ACKs and should know about the path b) it would be useless because packets could be lost after they passed the router c) the router would have to know what data were sent - a lot of states to keep!
2) Proprietary routers may not use Linux or BSD had have their own network stacks - which could be vulnerable or not. While SOHO routers may not get any update (many of them are already unsupported while sitting on a shelf to be sold), pro routers get updates until EoL.
Routers don't ACK anything but packets addressed directly to them.
But isn't that how NAT works? From the outside world POV, all TCP traffic is directed to the router. The router then extracts the "network address" from the packet and uses that to redirect the packet to the connected machine on the internal network that thinks (mistakenly) that it is connected directly to the rest of the universe? I would think that the NAT router has to handle ACK and other IP protocol stuff. But I could certainly be wrong about that.
In any case it is probably going to be severely stressed by a flood of tiny packets flowing through, and I'm guessing that it'll likely break before the "target machine" does.
I do not think NAT is terribly well specified. In particular the router might, or might not, buffer the TCP stream for the client. If the TCP is not buffered, then ACKs would be on the side of the client. But I would be curious to learn how it looks in practice on the typical and not-so-typical home routers.
In any case it is probably going to be severely stressed by a flood of tiny packets flowing through, and I'm guessing that it'll likely break before the "target machine" does.
Normal for cheap routers, ie limited memory space & CPU to handle & manage multiple sessions and any other processing thrown at it. Used to be a common problem with trying to use consumer CPE on shared connections given the actual number of sessions typically involved in visting a single website.
But SYN/ACK is a TCP thing. Rather than waiting and ACK'ng every packet, SACK ACK's every Nth packet, reducing latency and the classic LFP (Long, Fat Pipe) problem where with plain SYN/ACK, throughput suffers badly as latency increases. I've not looked at how this attack works but it seems a variation on resource depletion.
That works even without SACK - see TCP congestion/flow control, sliding window, slow start, ecc.
What SACK allows for is to avoid to re-transmit everything from the last valid ACK if only part of a transmission was lost. If A sent packets P0 (ACK) P1, P2, P3 (ACK?), and P2 was lost, with SACK instead of asking for the re-transmission of the whole P1, P2 and P3 you can ask to get P2 data only.
That works even without SACK - see TCP congestion/flow control, sliding window, slow start, ecc.
That.. depends, ie whether the implementation is RFC compliant, and which RFCs it may comply with.. But theoretically, should be based on RFC2018, which says..
The data receiver awaits the receipt of data (perhaps by means of retransmissions) to fill the gaps in sequence space between received blocks. When missing segments are received, the data receiver acknowledges the data normally by advancing the left window edge in the Acknowledgement Number Field of the TCP header. The SACK option does not change the meaning of the Acknowledgement Number field.
with the added fun of working out which packets you might be missing. Or getting a host (not router) to generate SACKs for non-existant data, or potentially exploit via the timers, or this bit-
Note that the data receiver is permitted to discard data in its queue that has not been acknowledged to the data sender, even if the data has already been reported in a SACK option. Such discarding of SACKed packets is discouraged, but may be used if the receiver runs out of buffer space.
And especially if you're trying to tune hosts for congestion control etc.. Common problems arise when flow control is 'wrong', or doesn't take stuff like LFPs, sliding windows etc etc into account.. Which can sometimes easily diagnosed by sending the same file via FTP (TCP) & then TFTP (UDP). If TFTP gives you the expected throughput, then your TCP needs more tuning. There can also be fun with packet re-ordering & asymetric routing confusing tuning algorithms, which can be common on transit networks given ISP's can't generally influence traffic flows bi-directionaly, so there may be differences in latency, packet loss etc in each direction.
(T'other common one is customers not realising MTU sizes vary by transport, ie smaller MTUs on L2 Ethernet VPNs, and IPV6 making that issue even more FUN!. For v4, pointing out you only really have to transport 576byte packets is technically accurate, but unlikely to result in a happy customer or sales. Even if max supported MTU is specified in contracts and SLAs. Oh for the good'ol days where website problems could be diagnosed by just asking if they were using 3Com.)
No, it's not how NAT works. NAT doesn't ACK, because it has no control of the transmission, nor it does anything about the state but the address:port mapping - only the two ends know exactly what has been sent and what has been received.
Say A sends a packet P to B through router R. If R ACKs P, A thinks B has received it. But if the packet is lost between R and B, (there could be other routers, switches, access points, etc.) and B doesn't ACK it, R can't resend it because it has no memory of packets, and B would never resend it because it has seen the ACK.
While it is true that NAT breaks the end-to-end design, and externally it's seen as the "host", is also true that it can't (and must not) perform the full TCP processing because it has not the required data to do it (and it would also make it far slower). Some protocols may require additional processing to work, and require the router to mess with parts of the payload, but that's why NAT is "evil", still, it doesn't Control (the C in TCP) the transmission.
The only thing NAT has to look at the TCP/UDP layer are the ports - and if and only if PAT is used, you can do 1:1 NAT between IP addresses and ignore ports - because that's what is used for mappings.
Fragmentation is handled at the IP level, and it's transparent for TCP and UDP (and with IPv6, routers don't do fragmentation either).
If sending a given stream of packets triggers other issues is a different thing, it doesn't depend on the SACK implementation. I don't think this attack needs a huge number of packets, anyway,
No, it's not how NAT works. NAT doesn't ACK, because it has no control of the transmission, nor it does anything about the state but the address:port mapping - only the two ends know exactly what has been sent and what has been received.
Slight caveat: Basic NAT shouldn't get involved, but devices like firewalls, caches, load balancers etc etc can & do end up manipulating or proxying TCP sessions.. Which I guess is important in this context, because those devices could then be vulnerable to this exploit.
A firewall/IDS does inspect packets but does not act like one of the hosts, and doesn't answer on their behalf (unless it closes a connection). Even if they are able to rebuild a session to inspect upper layers data, they still do it without being part of the transmission.
A proxy does - as it name implies, you talk to the proxy, not the end host - and would be vulnerable.
But a proxy usually works at the application layer, not at the TCP one. You have two end-to-end transmissions, not a single one, from a TCP/IP point of view. A caching proxy may not even need to setup up the external connection.
"1) Routers don't ACK anything but packets addressed directly to them."
However, most modern routers/modems/switches have some kind of web interface (or ssh, or even telnet), which is often exposed to the outside world, and will presumably be affected by this SACK flaw.
Unless of course your device uses a kernel so old that it's not affected.
> /proc is an internal pseudo file system, not a directory on the disk so modifications to /proc and its kids won't hang around when the system is rebooted.
Thats the only downside but simply edit your boot scripts or add a cron job that runs at boot to echo 0 to the file every boot.
I saw someone mentioned /etc/sysctl.conf
That sounds like the best place to preserve this parameter at boot. You'd want the parameter set as early as possible during boot so my cron idea would be a last resort as youd have to hope someone dont crash your machine while you are starting up cron.
You can also create an init script and link to it in /etc/rcS.d
Since they're unlikely to be running any sort of server or be accessible behind multiple layers of NAT who cares?
Well.. Given the way some apps seem to work (and offload personal data at the whim of the other end), I'm pretty sure there's some level of server stuff on many of them. And that's before we get to all the malwaregames and 'helpers' and the like that get installed.
Also, with IPv6 out there, well NAT is no longer any protection when devices are expected to always be directly routable regardless of whatever firewall/protections I want on my network (must look at how to block all IPv6 traffic at the entry point - the building can have a a trillion outside-facing IP6 addresses for all I care - just so long as nothing inside is reachable without my say-so (including F&F phones etc) I'm happy).
Since they're unlikely to be running any sort of server
Citation needed.
I just checked my phone, and all the sockets in LISTENING state are Unix-domain, not TCP. But I have no idea if that's typical, or how common it might be for Android apps to open listening TCP sockets.
(And NAT doesn't help if the attacker is on the same network. Coffeeshop attacks are plausible, if only for mischief.)
But I have no idea if that's typical, or how common it might be for Android apps to open listening TCP sockets.
I guess anything that listens for PUSH notifications - stuff like Skype or other VOIP/chat/message apps that can receive messages (sure they can poll the server for new stuff, but they'd have to do it quite often to get messages in a 'timely manner'[1] - meaning a lot of excess data) would effectively be running some level of "server" at least to the extent that it listens for traffic it did not initiate and responds to that traffic.
That would certainly be enough to allow this attack to come through.
[1] For me, generally if it takes 20 minutes to get and read your message so what? Even if it's important, it's likely I can't do anything about it for now. For most people, a delay of .000001ms for a "Hi honey. how's work today?" is an absolute disaster of Biblical proportions requiring lawsuits and C-level job losses.
"Since they're unlikely to be running any sort of server or be accessible behind multiple layers of NAT who cares?"
TCP isn't just for servers. If your device communicates over TCP, it is ACKing packets. As already explained by several people, NAT (and other forms of routing) are irrelevant to this.
"Did you read the article?"
Don't be so condescending. Not everyone understands how TCP works. You appear to know almost nothing about it. But yes, I did read it. And the Redhat vuln explanation. And understood it.
If this was happening on Windows 10 would the comments be so blase?I don't fucking think so.
Given the frequency with which winduhs crashes, and how often updates knock out the network etc, who'd notice?
Even if the updates didn't crash windoze, the multi-hour turnaround time for shutting down and restarting several times for even minor updates means the chances of a winblows machine even having a chance to do anything on the net other than suck massive loads of bandwidth to download even more system-hosing updates is vanishingly small.
I've upated half a dozen Linux machines since my first post in this thread. Only two needed rebooting, only one has had a reboot (the other will do so when it's owner goes to bed tonight). None of the other users even noticed anything - they still play their games or browse or stream etc without so much as a pause in activity. All updates are finished. When they turn the machines off tonight there's no 'Configuring updates. Please cross your fingers, go away for several hours, and hope&pray your machine restarts again', it just shuts down. When they turn them on tomorrow the updates are done, there's no "Fucked it up again. Reverting changees... Oh look new updates to install! No fucked it up again. Reverting changes.. Oh look! New updates!" loop to get stuck in, nor a 40minutes+ "installing updates" delay to wait around for while their productivity for the day goes out the window.
MS got it backwards - it's supposed to be "Handle major updates without even minor aggravation" not "handle minor updates with MAJOR AGGRAVATION"
(Now which icon... Am I trolling? Flaming? PTSD-angry??
Given the frequency with which winduhs crashes, and how often updates knock out the network etc, who'd notice?Well in my case that frequency would be never so I'd notice.
My W98 and my XP machines are among the most secure and crash-proof machines in the world these days. And no, there is no running firewall on them.
Ok, so they haven't been turned on in a very long time, but still...
I'm a really cheap gaming addict. I remember with fondness the later days of my Win98 partition--virii simply would not run on it.
I think SOASE:R is the most modern game I play with any regularity (I have purchased a couple of others ('Jupiter Incident' and 'Nexus' I think), but never actually played them). I still have Carrier Command in a playable state and play that through fairly often. (You can fit Carrier Command, several save files, and the OS you play it on all on a bootable 360K floppy - if you can find a machine that handles that!)
I'm such an 'elite gamer' that there is no one left alive who even remembers the ancient games! :)
Oh, man, I LOVE CC! I've got a supposedly legal version of it I can play on DOSBox. I say "supposedly", because there is a bug(?) wherein once you hit the limit of base modules to produce the first time, it won't produce more. So... I set the limit to 99 & win from there...
SOASE:R? Don't recognize that one.
The linux boxes you mentioned probably run 10% of the games the Windows ones do, support fewer hardware configurations and are less of a target than Windows PCs purely due to market share.
I'm all for Linux but being a blind fan boy of any product isn't a great idea, if/when (very much if) Linux really hits mainstream use it'll be the top target, we've all seen how Mac OS has gradually been more of a target even after making the big switch to BDS code base years ago, as Linux becomes more popular it'll just be more of a target and Torvalds will have more epic screaming sessions.
The linux boxes you mentioned probably run 10% of the games the Windows ones do, support fewer hardware configurations and are less of a target than Windows PCs purely due to market share.
So... All those complaints about how Windows no longer supports hardware that's not that old - they're all just holding it wrong or something?
Windows is so far gone even MS is abandoning it. Why else would they be going to such efforts to drive everyone elsewhere?
Sabroni, we're somewhat blasé about it because it's just another minor bug which was ALREADY PATCHED when ElReg ran the article. In fact, my users (corporate, academic and home) were already patched before I posted my comments late last night. It was no big deal, discovered & fixed in a matter of hours.
Bugs are inevitable in large coding projects. It's not the existence of bugs that are the difference in perception. It's the mechanism for acknowledging and dealing with them.
all sorts of gear from network or internet-connected TVs, routers, thermostats, light switches, CCTV cameras, and robot vacuum cleaners, to servers, PCs, smart fridges, dialysis machines, car infotainment systems, tractors, construction equipment, and uranium centrifuges, and so on,
power statins and electricity distribution grids...
https://www.nytimes.com/2019/06/17/world/europe/russia-us-cyberwar-grid.html
always makes me smile when these vulnerabilities that have obviously been around for years are suddenly a problem :) or are we expecting another list of leaks from certain intelligence agencies. As far as i cam remember TCP flow control and network protocols/file systems have always had these features.
thanks reg for keep us informed.
This came down today.
linux (5.0.0-17.18) disco; urgency=medium
* Remote denial of service (resource exhaustion) caused by TCP SACK scoreboard
manipulation (LP: #1831638)
- SAUCE: tcp: tcp_fragment() should apply sane memory limits
* Remote denial of service (system crash) caused by integer overflow in TCP
SACK handling (LP: #1831637)
- SAUCE: tcp: limit payload size of sacked skbs
I love Linux...
" [ Ubuntu: 4.15.0-52.56 ]
* Remote denial of service (resource exhaustion) caused by TCP SACK scoreboard
manipulation (LP: #1831638)
- SAUCE: tcp: tcp_fragment() should apply sane memory limits
* Remote denial of service (system crash) caused by integer overflow in TCP
SACK handling (LP: #1831637)
- SAUCE: tcp: limit payload size of sacked skbs"
No panic, no forced reboot.
1) You can choose when to reboot with Windows too. No reboot is ever forced. Especially on servers. You have even group policies to set it. On desktop you usually get reminders exactly because the system is not patched and thereby secure until you reboot.
2) Loadable kernel modules has nothing to do with that - it's in the TCP/IP stack, not in your network card module.
You'd need hot patching, but you need it enabled, and know very well what's running.
Keep on thinking you are secure while you're not - one day you'll learn the truth the hard way.
@AC
"You can choose when to reboot with Windows too"
Man, I must have been seriously hallucinating the numerous times I have witnessed Windows (including W10) reboot without warning (or my favourite W10 trick - advising you that Windows need to reboot in less than a minute and then promptly dying before you have finished reading the warning). Even if you have set every parameter to prevent that from happening; including setting it to only update and reboot only after hours.
==============> Not me; you.
1) I have seen Windows, including Windows 10, reboot WITHOUT authorization far too many times to count. Even when told specifically not to. Even the so-called "enterprise" versions. But I wasn't talking about Windows. I was talking about the OP's comment, and didn't even mention an OS.
2) I was responding to the OP's seemingly misguided opinion that any change to the Kernel means a reboot is necessary. I was not specifically commenting on the issue at hand.
Hotpatching is as hotpatching does. I use it occasionally. It's not like a reboot is going to kill me or anything ... I gave up on the uptime DSWs a LONG time ago.
I am perfectly secure, thank you very much :-) Oh, wait, you mean this computer attached via TCP/IP to the world-wide network called "The Internet"? No, that's not secure at all. No such computer is, for the simple reason that TCP/IP is inherently insecure. So I take precautions when using it, like not using my real name/address, not giving out my SSN, not doing any banking and not using my credit cards over it. Yourself?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
