TCP/IP has been multi-path from the git-go.
It's the way we designed it.
Seriously, learn about networking & history before trying to report on it.
When the iOS 7 incarnation of Siri was caught using Multipath TCP (MTCP) earlier this month, there was much excitement at it heralding a new era of communications. Which it may well do even though Siri was singing many hours before the sun begins to rise. A quick MTCP primer: TCP connections usually travel over one path. This …
Beat me to it.
May be the problem with a phone it that the mobile and wi-fi are given different IP addresses and your phone would have to add a simple virtual host/router to use both interfaces simultaneously. However, my netbook has both on at the same time, not sure how it decides.
>my netbook has both on at the same time, not sure how it decides.
Well each interface will have been given a weighting/priority and the comm's stack will decide which interface to use based on that weighting.
The advantage of the phone is that applications can use it's phone number (or other number personal/unique to it) to permit the remote application to tie the differing streams together - I suspect that a limitation with Apple's implementation is an assumption that only one instance of FaceTime or Siri is running on the device.
With regards to having wired and WiFi on a Windows machine at the same time....
If you go into control panel to where you see your list of network adapters (not describing the path because it varies by Windows version) then go to the advanced menu (alt and N if you can't see the menu) and then Advanced Settings, you can see the list of connections and the order they are used in. Priority is top down, so the connection at the top is used first if connected.
Presumably a similar option exists on Macs and *nix OSs though I'm not sure of what/where off hand.
@jake, err, no it hadn't
Sure enough, IP has always been, and is now, multipath. TCP session, however, is defined by the quadruple of source and destination layer3 addresses, and source and destination (layer4) ports. For a host with multiple interfaces that means one TCP session can use only one interface (at least, the incoming packets can only arrive on one interface). What they are talking about here is having different packets with different pairs of source and desitnation addresses belong to a single TCP session. Standard published in January 2013: http://tools.ietf.org/html/rfc6824
"To connect to that web site you ran up single-handed in 1983?"
No. Back then it was FTP and/or telnet..
MUDs, MUSHs and MOOs, on the other hand ...
I won't go into Usenet, that would probably only confuse you.
Hi Jake,
How did you get your multipath TCP sessions, that use multiple interfaces, that you've been doing since 1985, to pass stateful firewalls?
You know, after they became popular/mainstream, maybe ~94 ?
Thanks,
I suspect some people here are missing the fundamental difference between an application managing multiple single TCP sessions in software and treating them as one, and actual Multipath TCP in the network stack on the OS presenting a single stream up the stack.
The functional difference is like you putting a conversation together from hand written letters and e-mails that are all intermingled, and someone doing this for you and presenting you with the completed conversation.
In the first case you (the application, or essentially anything above Layer 5 in the OSI model), know who the data came from and that the data has come to you via different routes and you also have to reassemble to the data in the right order.
In the second case you just get handed a complete data set and you know who it's from, but don't necessarily know or even need to know how it got to you.
Ummm no. Just no.
Admittedly there are certain situations where the OSI model isn't accurate or doesn't apply, but it is still the fundamental core of networking particularly TCP/IP based networking (which is the majority of all networking).
It is particularly relevant when comparing TCP an Multipath TCP. If you don't see how I suggest you go and get some education/training. I'd suggest a CCNA, but frankly it sounds like you may need to start out a little more basic than that.
"Thus, TCP/IP has always been multi-path."
Only once you leave the source host. These days, however, getting from your source host to the first hop is the least reliable part of the connection.
Yes, TCP packets may be carried across multiple routes once they leave their source IP address, but: TCP connections must have one and only one endpoint address at each end. You must choose the source and destination IP address before you initiate the connection, and you're stuck with this choice until you close the session. If one of those endpoint addresses becomes unreachable (e.g. a Wifi link fails), then the whole session fails, even if the host you were communicating with still has other viable connections.
Multipath TCP allows you to carry a TCP session with multiple interfaces at each end. A master Multipath TCP flow is established, and it delegates packets to one or more additional "plain-TCP" "sub flows". The choice of which sub-flow receives a particular packet follows the whatever TCP congestion-control algorithms you've chosen, so a slow link is not given as much traffic as a fast one, for instance. However, individual sub-flows may drop or join the master flow without bringing down the overarching Mutlipath TCP session. A crude analogy is that MPTCP is like placing the first-hop router into the TCP stack of the local host (MPTCP isn't a routing protocol, though, it's cleverer than that).
This isn't bonding. It's not load-balancing. It's not failover. It's TCP, but with the session's packet stream spread across two or more TCP links. As a result, when a MPTCP subflow goes, the overarching session continues, albeit at a lower bandwidth: just like the whole session doesn't collapse when a single packet is lost in TCP.
This is one of the killer features of MPTCP, and it's the one our industrial customers really benefit from: try running a Citrix session on a single, patchy mobile connection that keeps collapsing the TCP connection, and you'll understand how useful it is.
The second big win is that you create a TCP session that has the aggregate bandwidth of all your available interfaces available to it. This is what we're pitching to residential and small-business customers here: http://igg.me/at/mpn - but you also get the reliabliity too, because at heart, it's still TCP.
And that's the point: MPTCP is not trying to replace TCP; it builds on TCP and extends it to make better use of hosts with multiple interfaces.
Well the use of globally unique network addresses (eg. IP address) for all end nodes in a network would greatly facilitate MPTCP communications between them. But I wouldn't count on NAT or IPv4 going away anytime soon, so any viable MPTCP solution needs to take account of NAT (as should IPv6 - but that is a totally different debate).
It COULD fix it. But IPV6 does not deny the use of NAT. And a lot of people like the fact that their actual IP address and location is known only to a big mobile ISP NAT table, and not to any joe soap with malicious intent on the far end of a site they stumble upon.
IPv6 does indeed solve the NAT issue from a routing perspective. That said, one to one "NAT like" address mapping (in your stateful firewall) is sooooo tedious to do manually. Even a decent sized home network with 10 or 15 devices can be troublesome if you want to grant different rules to different users. Granted, many will probably find in the end that "IPv6 NATing" actually works better than creating blanket rules for a particular subnet or whatever, given the obvious granularity that becomes possible. Unfortunately, it'll be a while before decent automation for this task reaches the unwashed masses (you, me, SMBs, schools, etc.), and large enterprises aren't likely to lead the way either as size=inertia in my experience.
Battery life?
Granted, my phone is a gently aging Nokia N900 (and could probably do with a battery change anytime soon), but given the propensity for data-hungry connections over 3G, and almost anything over WiFi to consume battery power like a starving dog in a butchers shop, isn't this generally a bad thing?
Mobile devices are, by nature, limited in show long they can run due to the big chemical block sitting in the back. Having multiple radio emitters firing up and connecting all at the same time can't be doing wonders for the usage time figures. Pretty serious thing, when your battery is glued and screwed into your block of shiny fruit-plastic.
With the right resources, it's a good plan. But until we have usage time on battery into the 'days' figure, it's not going to do much more than sell a few more phone chargers, so people don't run out of juice halfway through the day.
Well, while the rule of thumb is power = x * data rate * time, I'm not sure if that will be the limiting factor here.
A worst case scenario might be watching HD video which is using wifi + 3/4G. Simply keeping the screen lit might well be the limiter here.
It sounds to me like the approach is an attempt to anticipate the long anticipated merge of mobile phone technology and ethernet where everything becomes IP. This might make it easier for applications to switch between the two without restarting connections. Applications for this would be things like OTT voice and video communication where the latency of restarting the connection would be noticeable. But I'm not sure if they aren't already quite good at this: I've had a video call with someone on Hangout moving from their wifi to mobile data.
From what I know of 3G I don't see this really happening a lot as there's just too much going on in the carriers' networks. In 4G it's a no-brainer and there are probably already attempts to manage it in silicon - I'm not sure application should be getting the choice - where the phone presents a single interface to a device which might in reality be two or more physical interfaces. Isn't this how MIMO works?
I just wandered into the house after starting a Facetime conversation on 3G and it seamlessly transferred to WiFi (It must have done as I have no signal in the middle of my house). I know this doesn't mean it is multi-tunneling, but I do wonder if Facetime is also using this tech? I know that that doesn't work with Skype, it cuts off as soon as I walk into the house and I have to re-establish over WiFi.
Yep typical Reg hatchet job without all the facts. Before even understanding Apple had implemented multi-path TCP in iOS 7 I had noticed the following.
Before, when using iOS 6, I encountered a problem when leaving my apartment. The journey through my front door would take me out of WiFi range, then, as I walk past the front of my apartment, I would come back into WiFi range (but hovering on the edge of signal sufficient strength) and then after a 30 yard stretch, fully back out of WiFi range. This was annoying because I found it a frequent pattern that if I left the house in a hurry, I would want to use Siri to make a call or send a text ("I'm in my way," or "I'm running late" etc) but invariably and annoyingly, with the WiFi --> 3G --> wiFi --> 3G transitions going on the request would fail. I noted, even before reading about the muti-path implementation, that since installing iOS 7 I tried it expecting it to fail, but it didn't. I have tried multiple times since and it has worked every time. Being technically minded, I was left wondering how there could have been such an improvement, and only later read up about the multi-path TCP implementation.
But more than that I'm now using, FaceTime audio at every opportunity because the sound quality is so much better, FaceTime audio also doesn't fail while making the same transitions. Now that is a pretty good real world test and when you have an understand of the challenges, it is impressive.
Beyond that the several FaceTime audio conversations I've had with my girlfriend when she is travelling in her car (it works via hands free) have also worked reliably without dropping out (my natural inclination would be never to attempt that kind of connection if I knew she was travelling because cell to cell transition and signal lock for 3G is not as good as for GSM audio - and to be accurate this shows the Vodafone network must be quite good now for 3G rather than that multi-path TCP has an effect on cell transition - though the protocol may be a little more robust when dealing with temporary drop-outs. However it does illustrate the multi-path implementation is solving the problem of known WiFi networks popping up and taking over the connection (and often when a WiFi network is joined it is blocked from the network by a login screen). It seems driving past Costa and Starbucks is doing nothing to interrupt.
So in true The Register style, they are jumping the gun looking for theoretical negatives against a tech Apple have implemented first, before checking the real world performance. They did the same with their ill informed article on Apple 5s 64 bit processing, which has now been confirmed as ignorant speculative negative hokum:
http://www.mikeash.com/pyblog/friday-qa-2013-09-27-arm64-and-you.html
But the FaceTime app can achieve that handover without MTCP, although MTCP might make simplify the app. I wonder whether the app has knowledge of the WiFi and 3g/4g interfaces and hence effectively creates/maintains two TCP sessions using which ever it determines has the better service? It would seem from your experience that in iOS7 Apple have refined the algorithm for the handover between the two sessions.
Also from your experiment it would seem that iOS implements both client and server functionality.
Facetime already binds Wifi and UMTS connections, and as it's a UDP application (well, the audio/video streams that account for the bulk of its traffic are UDP), Multipath TCP has no real bearing on any performance improvements you may have seen.
A change to how connections are chosen would be enough to give the improvement.
@Kristian Walsh
I think that misses the role TCP plays. While UDP would be the protocol for streaming the improvements must relate to connection, re-connection, and stream selection/control, which surely involve TCP. I now need to go and read up on strategies for consuming a UDP packet streams via multiple access links - but that is implementation logic for how it is done in practice and is subservient to by the bigger problem of stream set-up and ongoing QA/control.
"But the FaceTime app can achieve that handover without MTCP"
True, but I expect, since they have implemented MTCP, it will be involved, and then then as more servers become MTCP aware, other services will also benefit. I would not be surprised though if there are tweaks outside of the standard, we can only speculate, but I think a combination of MTCP and other tweaks is indeed likely. Additionally I have a suspicion, and will need to do more to confirm, that they are using the 3G channel to overcome double NAT connection problems. My girlfriends home office suffers from a double NAT problem with FaceTime. I have to wait for her to call me back if I try to call her with FaceTime on her office machine as FaceTime has a problem making the connection the first time - a typical symptom of a double NAT issue. I've noticed this "try to connect twice" problem seems to have resolved itself if I am placing calls to her iOS7 iPhone. Her phone connects to the same WiFi network and had the same problem as her desktop Mac when it was on iOS6.
"So in true The Register style, they are jumping the gun looking for theoretical negatives against a tech Apple have implemented first, before checking the real world performance. They did the same with their ill informed article on Apple 5s 64 bit processing, which has now been confirmed as ignorant speculative negative hokum:
http://www.mikeash.com/pyblog/friday-qa-2013-09-27-arm64-and-you.html"
Don't come around here with your 'facts', young man. This is The Register, not some tawdry fact-reporting website.
Hi @ian 22,
That will be an interesting case but I'm pretty sure it will still fail. MPTCP protocol requires an MPTCP aware server to work. Siri and FaceTime we can expect to be compliant. You can be pretty sure for now pretty much all general internet radio servers won't be though. iTunes radio once it makes it over to Blighty will probably be made MPTCP compliant as the data will be streamed from Apple servers.
I wonder how long it will take before the average Apache host is compliant and if it will be possible to ensure MPTCP can be run on servers where you don't have admin control over the IP stack?
The default right now is for smartphones to use wifi whenever possible, hard to see what objections they'd have to more data going over their networks instead do wifi.
I think you're making too much of this though, it's on Siri for performance reasons, they're very unlikely to enable it for the rest of the phones traffic for the obvious bill shock reasons you outlined.
Right now it is only used for Siri, which uses very little bandwidth. They'll obviously have to add some end user control if they decided to implement it for something that uses a lot of bandwidth like iTunes Radio or Maps.
I suspect they did it for Siri because they wanted to improve the response time in situations where you may be on a crappy public hotspot but have a great 3G/4G sitting at your disposal. You don't want to have a wait an extra second for a response if you don't have to, but if it took an extra second for a new Maps area be downloaded as you scrolled around it won't bother you too much.
TCP is a connection protocol carried on the IP datagram connectionless protocol. In the original theory it doesn't matter how many parallel routes are used for the IP datagrams.
Large networks often load share by sending IP datagrams down internal parallel paths. When they arrive at the destination, intermediate or final, then a TCP-aware stack has to re-assemble the packets into the correct sequence before handing them to the application.
The behaviour of the sending and receiving TCP stacks can then get interesting. Some devices have been seen to ignore any out-of-sequence packets - causing many resends. Others have made immediate, unnecessary requests for a resend of a "missing" packet in order to boost the throughput. Unfortunately both these behaviours can can lead to the sender thinking there is network congestion - and it slows down its subsequent transmissions to a crawl.
TCP-aware firewalls etc just complicate the issue of using multiple paths unless it is a confluence point for all the routes.
This only makes sense on a phone if the phone can use TWO different Mobile bands at once. Then you can get more speed.
Variations of this have been used in specialist Routers for some time before the recent spec, such as 2 or more ADSL Modems when multiple phone lines exist but speed is only 1Mbps to 3Mbps due to distance. The ISP then supports it and the LAN devices and Internet Servers don't need to know.
But it illustrates the cluelessness of Apple to implement this between WiFi & Mobile unless there is an option to turn it off. I seriously doubt the iPhone can make two simultaneous connections on say 900 UMTS and 2100 UMTS. bands. Two connections on the same band will in most cases give the same channel so give no net increase in speed or reliability.
So there are two use cases (aggregate multiple phone lines or multiple bands), but the iPhone doesn't do either. Also the problem of lack of server support vanishes if the ADSL ISP or Mobile carrier implements it. Such a specialist link aggregation is useful, but Multipath on the Public Internet between Mobile and Fixed is stupidity.
Switching between links (changing between Mobile and WiFi) is a different trick to true Multipath or Link aggregation. It doesn't need any NAT support at all or Multipath. It does require application support at the Server and Client. It can be done purely at the application level on the client and with some persistent object (cookie like) at the server so when client reconnects with a different IP the server Application regards it as the same session. Plain vanilla web servers of course can use a browser cookie.
No! this makes sense when a device can use two different network interfaces and paths, hence why it is 3G/4G and WiFi (ie. UMTS 1800 and 802.11 on 2.4Ghz). One form of usage of multiple mobile (or WiFi) bands is already part of those specifications, in the case of WiFi if you are using 802.11n. Aggregation is slightly different in that the user and the recipient (or man-in-the-middle ie. an ISP) agree to link multiple separate connections together, whether they be different mobile bands or physical phone lines.
What is interesting is whether in 4G devices, they have totally separate 3G/2G and 4G radios and aerials, hence are able to do as you say and concurrently use both. Obviously the 4G specifications have allowed for non-concurrent use, hence the reason for delays when a 4G device drops back to 3G for voice calls.
No it doesn't show the cluelessness of Apple, the handover between mobile (4G/3G/2G) and fixed (ie. WiFi/DECT etc.) has been a major stumbling block for years. It would seem that Apple have implemented a scheme which (for it's applications) minimises and facilitates the handover between services, so that a user can start a session at home over WiFi, walk outside and continue the session over mobile and conclude the session on arrival in the office over WiFi, so sorry I don't agree with your statement "Multipath on the Public Internet between Mobile and Fixed is stupidity".
@Roland
This article gives a good overview. It's by someone who has the right credentials so we can be fairly sure of the facts and confirms you are correct is good for 3G and WiFi connections. It appears they are using it purely for mobility purposes (efficiently maintaining a connection using multiple links - both 2.5G/3G/4G and WiFi) and not really for ramping up throughput. Other standards have been developed to do similar but rely on IPV6 which isn't yet commonplace enough, so they have exploited MPTCP for mobility effects.
http://arstechnica.com/apple/2013/09/multipath-tcp-lets-siri-seamlessly-switch-between-wi-fi-and-3glte/
"Other examples of exchanges that might be SYN-sensitive include:
A user establishes a VPN to the corporate network over the home WiFi. In the Multipath TCP world, it's feasible that traffic could also traverse the available mobile network – but it won't be secure."
I can't see how this is feasible. When you're connecting over a VPN, 99% of the time you're connecting to a DESTINATION IP that's private (RFC1918), ie, that destination doesn't route across public internet.
How would that traverse the mobile network? Surely it woudn't, shouldn't the mobile network drop RFC1918 destinations at the edge?
It certainly wouldn't get to it's destination, not without being encapsulated down the VPN tunnel, because the mobile network wouldn't have a clue how to route it to "your" 192.168.10/24 network at head office, as opposed to the one run by IBM, or whoever, in their office.
"With SSL VPN you should see no difference, since the entire VPN tunnel sits on top of the carrier TCP."
You're sure that SSL VPNs work "sitting on top" of Multipath TCP? You've tried that, have you? Through stateful firewalls?
If so, fair play, but I'm having trouble thinking through how that would work, exactly... Any recommendations for vendors who support this? I can think of zero.
"With IPSec VPN then either you would have separate VPN sessions for each network interface or be restricted to using only a single network interface."
If you're "restricted to using only a single network interface", how is it Multipath TCP?
Again, have you tried this with two separate nics, and two separate VPN sessions? I'd be incredibly surprised if any vpn endpoint's happy with the SYN packet for a TCP session arriving over one IpSec tunnel, then the ACK for the 3 way handshake for that session arriving down a separate IpSec tunnel.
If you've tried it, and it works, fair play, please let me know the vendor who supports that. Almost all I can think of will drop the ACK as out of state.
"With SSL VPN you should see no difference, since the entire VPN tunnel sits on top of the carrier TCP. "
This is nonsense. Give me an example of even just an HTTPs server that supports multipath TCP at present.
So I've got a phone with a wifi and 3g connection, wifi public ip of 50.50.50.50, 3g public ip of 60.60.60.60. I send a SYN down the wifi interface, get a SYN-ACK back, send the ACK down the 3g interface. You're saying I'll then see the cert exchange?
I'm saying I'll see fuck all, because there are no multipath SSL servers which are currently used, and the server will see the ACK come from a different ip than then SYN, and drop it as out of state.
So once you get past that issue, tell me an SSL VPN vendor who supports multipath SSL vpn termination. No?
In that case, this isn't feasible. If everybody implemented multipath TCP SSL servers and someone started selling a multipath SSL compatible VPN endpoint, it's feasible, but until then it's as useful as IP v5. Yes, I did mean v5.
If your iPhone is grabbing free WiFi as you walk down the street, instead of using up your lovely and relatively expensive mobile data connection, that's a revenue threat to address
But iPhones (and I assume all the other smartphones) have done this since the beginning: Preferring to use WiFi for data when available over cellular.
You seem to be wrong on two counts.
Plenty of businesses use RFC1918 addresses in the 10.x.x.x range for their internal wifi networks.
My phone is connected to the Orange/EE network and I have a 31.x.x.x address.
"An Android app developer could make cost inferences by simply looking at which physical interface is tied to which IP address (moreover, since mobile networks are still presented to the Internet as “great big NATs”, a 3G address that's bound to a 10-range IP address at the device is clearly attached to the Telstra mobile network). An Apple developer would have to wait for Cupertino to make suitable APIs available – if and when."
Automatically managing multiple IP connections has been done before. I worked on the Identiprise SecuredMobile product back in 2008, which routed data on a VPN dynamically between multiple interfaces and WiFi connections. The connections were handled transparently to the IP packets, which were routed to the appropriate interface. Cost metrics as well as connectivity were included in the routing heuristics. WiFi was prioritized over cellular or other radio connections.
Unfortunately, when the banks collapsed we didn't make another round of funding. Ah, well.
"This can be a bit of a problem — especially when the two NICs share the same parent network and you’re trying to preserve sane traffic flows. In a nutshell, this post will explain how you can ensure traffic going into eth0 goes out only on eth0, as well as enforce all traffic going into eth1 goes out only on eth1."
http://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/
Apart from spelling Geoff's surname wrong there are a few misunderstandings in the article.
RFC 6824 takes NAT into account and has a whole chapter (6) dedicated to middlebox interactions. Of course a sufficiently stupid middlebox can break anything, but MPTCP will then just fall back to regular TCP with a small delay. Keep in mind that each subflow is (almost) a regular TCP session for everyone but the two end hosts. One can probably assume that they tested this on at least the most common middleboxes out there.
The primary use case for Apple is probably resilience and not more bandwidth, i.e. being able to switch seamlessly between two otherwise unrelated connections and only being minimally affected when one disappears.
The chicken-and-egg part isn't much of a problem. We're talking OS level stuff so the application developer doesn't have to decide. And since everything is backwards compatible you can implement it little by little. Compare it to TLS for SMTP.
Regarding VPNs and security, MPTCP also takes this into account. A new subflow cannot send data before the other end has been verified as the original partner. Trying to create the subflows will leak information about the other end's address(es) but VPN software could prohibit this like any other NAC feature.