During the last weeks I have been assessing how Mobile operators mangle HTTP connections and today I start a series of posts to present the results.
One of the main differences between mobile and fixed operators is that the first have been providing the so called Valued Added Services (VAS), while the second, in most cases, are just fat pipes. The main VAS enablers are the former Wireless Application Protocol Gateways (WAP GW).
Ten years ago, WAP GW had the important role to convert the efficient and compact WTP/WSP to the standard and broadly used TCP/HTTP. Nowadays, when all (relevant) devices support WAP2.0, WAP GW are not much different from any enterprise Proxy. The main difference is that in order to support the VAS, WAP GW enrich the HTTP headers. There are different flavors of header enrichment: on one hand the most common behavior includes adding a user ID, like the mobile phone (MSISDN) for dedicated internal services allowing the VAS to identify the user, and most probably charge him/her. On the other, some operators would modify the HTTP headers for different reasons. Also some others would add extra information like bearer type, Proxy ID, device real IP, etc.
The methodology for the tests is simple. I have several GPRS/UMTS modems with the SIM cards of the operators that I want to analyse. I created a script that would initiate a connection through a given modem from one of the mobile operators and access a server I own that replies with the HTTP headers that reached the server. As I know the headers I input and the ones that reach the server I can compare them and note the results.
When doing the connection to the operator I do it in two ways. First I would use the so called WAP connection and connect trough the WAP GW. In this case I´m simulating a standard mobile phone like most Nokia, Motorola, Samsung, Sony Ericcson.... In the second connection I would use the so called WEB or INTERNET connection, simulating an Android, iPhone or standard PC with a data card. Although most people think that in the second case the user is accessing the internet directly, without going through a proxy we will see that it is not the case. Since the advent of the Androids and iPhones, which do not support an operator proxy, mobile operators have deployed transparent proxies (aka intercepting proxies) on the "Internet" connection, forcing also these devices to go trough them.
Having said that, lets take a look at some real examples from the french mobile operators: Orange, SFR and Bouygues. First of all, these are the original headers of my script:
TE: deflate,gzip;q=0.3
Connection: TE, close
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1
whereas when using the direct connection the headers were the same but the User Agent was:
User-Agent: HeaderValidator/1.1-dir
Below I add the headers as received in the server for the different connections. In red I highlight the unexpected changes, while in orange I marked the ones that where understandable for a proxied connection.
=== Orange France through WAP GW/Proxy ===
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1
X-Nokia-CONNECTION_MODE: TCP
X-NSN-PROTOCOL-TYPE: WAP2.0
X-Nokia-BEARER: GPRS
X-Nokia-PLTF: NBG3.1.3_PP
X-Wisp: wisp2
Via: 1.1, 1.1 proxy (proxy)
Cache-Control: max-age=259200
Connection: keep-alive
=== Orange France direct connection (INTERNET) ===
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1-dir
X-Wisp: wisp2
Via: 1.1 proxy (proxy)
Cache-Control: max-age=259200
Connection: keep-alive
=== SFR through WAP GW/Proxy ===Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1
X-vfprovider: SFR
X-vfstatus: 10
X-Nokia-BEARER: GPRS
serviceControlInfo: 18+=-
x-nokia-gid: 259107281784650
X-Nokia-CONNECTION_MODE: TCP
X-Nokia-gateway-id: NWG/4.1/Build79
X-Nokia-ipaddress: 10.187.207.132
Via: 1.1 proxy.cwg.net (proxy)
X-Forwarded-For: 10.187.207.132, 10.187.207.132
Cache-Control: max-age=259200
Connection: Keep-Alive
Pragma: no-cache
X-BlueCoat-Via: 28409D05D072730C
=== SFR direct connection (INTERNET) ===
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1-dir
X-vfprovider: SFR
X-vfstatus: 10
X-Nokia-BEARER: GPRS
serviceControlInfo: 18+=-
x-nokia-gid: 259107281784650
X-Nokia-CONNECTION_MODE: TCP
X-Nokia-gateway-id: NWG/4.1/Build79
X-Nokia-ipaddress: 10.205.32.74
Via: 1.1 proxy.cwg.net (proxy)
X-Forwarded-For: 10.205.32.74, 10.205.32.74
Cache-Control: max-age=259200
Connection: Keep-Alive
Pragma: no-cache
X-BlueCoat-Via: DFF4C47D046B9013
=== Bouygues through WAP GW/proxy ===
TE: deflate,gzip;q=0.3
Connection: TE, close
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
Host: m.mobiclip.net
User-Agent: HeaderValidator/1.1
=== Bouygues direct connection (INTERNET) ===
TE: deflate,gzip;q=0.3
Connection: TE, close
Accept: text/html,text/vnd.wap.wml,application/vnd.wap.html+xml,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/x-wap.wml,text/x-hdml,text/vnd.sun.j2me.app-descriptor,application/java-archive,application/octet-stream,,image/png,image/gif,image/jpg,image/jpeg,*/*
Accept-Charset: iso-8859-1,utf-8
Accept-Language: en-us,en;q=0.5
User-Agent: HeaderValidator/1.1-dir
In France we can see there is one operator, Bouygues that preservers the HTTP headers intact, probably having a better performance than SFR and Orange which add a lot of useless information.
Looking at the "new" headers added by Orange and SFR, the first question that arises is, do we really need to know the information below? What for?
X-NSN-PROTOCOL-TYPE: WAP2.0
X-Nokia-PLTF: NBG3.1.3_PP
X-Wisp: wisp2
Via: 1.1, 1.1 proxy (proxy)
X-vfprovider: SFR
X-vfstatus: 10
x-nokia-gid: 259107281784650
X-Nokia-CONNECTION_MODE: TCP
X-Nokia-gateway-id: NWG/4.1/Build79
X-Nokia-ipaddress: 10.205.32.74
Via: 1.1 proxy.cwg.net (proxy)
X-Forwarded-For: 10.205.32.74, 10.205.32.74
X-BlueCoat-Via: DFF4C47D046B9013
What is the purpose of wasting GW memory and CPU cycles adding this? Do we really need to know our device uses WAP2.0 or TCP? Or the Nokia GW ID or the Blue-Coat proxy ID? What for? As we will see in the future posts the variety of data across the main mobile operators is so big that we can conclude that this HTTP noise can´t be of use for anyone.
The surprising part is that, especially in the case of SFR, all the headers are also added by the transparent proxy in Internet connection. While in the case of Orange we see that the number of headers has been dramatically reduced.
I would assume that SFR is adding the transparent proxy "in front" of the standard Nokia GW, routing all Internet connections through it and later through the BlueCoat. Good for Nokia, as that sounds like they would need a lot of HW and licenses!!. While Orange France seams to follow a more sensible approach redirecting the Internet connections to the "WISP2" proxy which might be in charge of the useful header enrichment.
That´s all for now. Next time we´ll have a look at how the Italian Mobile Operators deal with the header enrichment.
Subscribe to:
Post Comments (Atom)
Thank you for your article.
ReplyDeleteI'm using Orange in france and encounter problems with synchronization with Exchange due of proxy.
When i use a different access point internet-entreprise instead of orange it works well.
So proxy behaviour is different with access point "orange" and access point "internet enterprise".
this is good picture
ReplyDelete19 This is very smart tutorial...thank you for sharing
This whole research is amazing! Is there a way to contact you? I have an operator-related question that I've been desperately trying to find an answer to.
ReplyDeleteHi,
ReplyDeleteyou can contact me at xufxuf @ gmail . com
Thank you for your article.
ReplyDelete