Calling Web Service method and GPRS

[Korwin]

Hi,
I am building an application that I would like to invoke a web service in
the background without interrupting the application from functioning
normally. I call Web Service method in a separate thread. On the server,
web service method waits for data change or when fixed time (10 min) elapse
returns response to mobile application. It works fine only when application
connects through ActiveSync, but when application connects through GPRS the
response not always returns. It looks like connection is broken and
application can't notice that. No exception is thrown. Could someone point
me what could be wrong?

Korwin

 

[corradolab]

web service method waits for data change or when fixed time (10 min) elapse
returns response to mobile application. It works fine only when application
connects through ActiveSync, but when application connects through GPRS the
response not always returns. It looks like connection is broken and
application can't notice that. No exception is thrown.

I understand your server is sending data to you handheld with a
stateless protocol (SOAP and HTTP).
This can fail anytime, because your handheld can move out of coverage
and/or change the IP address (because it change the "base station" it
connects to).
Basically the only "realiable" communication you can get over GPRS is
when the handheld call the server, and even this way you must be
prepared to handle a loss of connection at any time.

Kind regards,
Corrado

 

[Guest]

I would expect a SoapException to be thrown in this case but I'd say this is
very bad design as you are relying on a constant connection which you should
never do in mobility solutions.

Out of interest, are you using a worker thread or using the built in
asynchronous model that web services support?

A better solution is to use raw sockets ontop of TCP over web services
(depending n what you need to achieve) as this allows greater control in a
connection oriented model. It allows guranteed message delivery.

 

[Korwin]

Out of interest, are you using a worker thread or using the built in
asynchronous model that web services support?

I call web service from separate thread synchronously and on server I'm
using AsyncCallback and IAsyncResult to delay the response.

A better solution is to use raw sockets ontop of TCP over web services
(depending n what you need to achieve) as this allows greater control in a
connection oriented model. It allows guranteed message delivery.

I need to create mechanism similar to AUTD (always-up-to-date) or
DirectPush. It means I want to minimize the network traffic and costs
by calling web service and delay the response.
Is this possible use sockets and we services and the same time. I mean
connect to server and checking the connection using sockets and somehow call
webservice through this connection.
I understand it could be impossible, but I have faith.

Korwin

 

[Guest]

Depending on your architecture, have you considered writing a server that
listens on the device, then your back office pushs out the information when
required. This is how a system I wrote recently works. This would save you
the need to have a permanent connection to your web server.

It is possible to call a web service from a socket server no limitations here.

 

[Scott Gifford]

Hello Simon,


[...]

this is very bad design as you are relying on a constant connection
which you should never do in mobility solutions.

I would be very interested in hearing your thoughts or experiences on
this design. I'm in the middle of writing an application with just
such a design, based on recommendations from Microsoft's Exchange
always-up-to-date (AUTD) design here:

http://msdn2.microsoft.com/en-us/library/aa446486.aspx
http://msexchangeteam.com/archive/2005/06/07/406035.aspx

I had assumed that since this was what MS used in Exchange, it was the
best approach available. I'm just starting testing my app, so I don't
have much experience on the real-word reliability. Any experiences
you can share would be appreciated.

Also, elsewhere in this thread you wrote:

Depending on your architecture, have you considered writing a server that
listens on the device, then your back office pushs out the information when
required. This is how a system I wrote recently works. This would save you
the need to have a permanent connection to your web server.

In your experience, does this work over most carriers' GPRS networks
(no firewall issues, etc.), and does the device wake up properly when
it gets a new connection?

Thanks!

-----Scott.

 

[Guest]

Sorry for the late response - I have been away lately.

I don't like the way Exchange handles push email - there are too many
constraints. I would go as far to say I think it is a bit of a kludge and
somehting that was done to quickly rival Blackberry devices.

Implementing this type of functionality AUTD - a constant connection with
back end server via a mobile device is a big no no in my experience. Bearing
in mind I'm talking about enterprise application development which is scaled
to 1000 - 2000 devices and in some cases more. Another problem with AUTD
using HTTP Web Services with Windows Mobile, is reliability. Unless you are
using Web Service Enhancements (WSE) with reliable transactional support Web
Services are not good for LOB systems. WSE is not supported in CF2.

My advice is to design your mobile app with the assumption that connectivity
will not be available and every message you send will fail and only connect
when required send the data and then disconnect. This makes the system very
very scallable, reduces downtime and reduces bottlenecks unlike AUTD. The
patterns and practices team has written quite a lengthy paper on this subject
- occasionally connected smart clients. It is important your application will
know when a message didn't reach it's destination - without transactional
support and message tracking - this is difficult.

I have found using Web Services on mobile devices ok for silly simple small
scale apps, but I would not recomend the AUTD approach for LOB large scale
apps.

See here for the patterns and practices paper:
http://msdn2.microsoft.com/en-us/library/ms998482.aspx#scag-ch04_topic3

--
Simon Hart
Visual Developer - Device Application Development MVP
http://simonrhart.blogspot.com

Hello Simon,


[...]

this is very bad design as you are relying on a constant connection
which you should never do in mobility solutions.

I would be very interested in hearing your thoughts or experiences on
this design. I'm in the middle of writing an application with just
such a design, based on recommendations from Microsoft's Exchange
always-up-to-date (AUTD) design here:

http://msdn2.microsoft.com/en-us/library/aa446486.aspx
http://msexchangeteam.com/archive/2005/06/07/406035.aspx

I had assumed that since this was what MS used in Exchange, it was the
best approach available. I'm just starting testing my app, so I don't
have much experience on the real-word reliability. Any experiences
you can share would be appreciated.

Also, elsewhere in this thread you wrote:

Depending on your architecture, have you considered writing a server that
listens on the device, then your back office pushs out the information when
required. This is how a system I wrote recently works. This would save you
the need to have a permanent connection to your web server.

In your experience, does this work over most carriers' GPRS networks
(no firewall issues, etc.), and does the device wake up properly when
it gets a new connection?

Thanks!

-----Scott.

 

[Scott Gifford]

[...]

only connect when required send the data and then disconnect.
[...]

See here for the patterns and practices paper:
http://msdn2.microsoft.com/en-us/library/ms998482.aspx#scag-ch04_topic3

Hi Simon,

Thanks for your reply, that paper was very helpful!

It seems that connecting when required, sending the data, then
disconnecting is ideal for when the client is generating data and
sending it to the server.

However, if the server is generating data and sending it to the client
(for example push email), it seems to me that this approach basically
becomes polling the server, and it's much harder to know when to
connect. In particular, if your goal is to receive messages from the
server as quickly as possible, you're stuck with frequent polling
(which uses more power), or longer delays.

AUTD takes advantage of the GPRS radio waking up the device, and so it
can basically initiate the connection, then go into suspend mode until
and unless notifications are available.

Of course, reality is more complicated (for example keepalive messages
must be sent to keep the connection up), but I think the basic
tradeoffs remain.

What are your thoughts on a system in which a Windows Mobile client
must receive messages from a server as quickly as possible?

Thanks!

---Scott.

 

[Guest]

Hi Scott,

It seems that connecting when required, sending the data, then
disconnecting is ideal for when the client is generating data and
sending it to the server.

However, if the server is generating data and sending it to the client
(for example push email), it seems to me that this approach basically
becomes polling the server, and it's much harder to know when to
connect. In particular, if your goal is to receive messages from the
server as quickly as possible, you're stuck with frequent polling
(which uses more power), or longer delays.

You are right about this if you haven't implemented true push technology.
However, if you have implemented true push technology, ie: a server process
listens on the device waiting for the back office to connect to it, once
connected, pushs out the message then disconnects, then you will remove the
need for polling.

AUTD takes advantage of the GPRS radio waking up the device, and so it
can basically initiate the connection, then go into suspend mode until
and unless notifications are available.

This is true in a true push type environment as well. Of course you would
need to connect to your APN via GPRS/UMTS when your application starts so
that the back office can see the device.

Of course, reality is more complicated (for example keepalive messages
must be sent to keep the connection up), but I think the basic
tradeoffs remain.

This will be a much more expensive approach to using true push technology.

What are your thoughts on a system in which a Windows Mobile client
must receive messages from a server as quickly as possible?

As I said above, you will always need a GPRS connection to your APN, but
with true push technology the only data going up and down the wire is the
data you want to send receive. No polling or keep-alives or checking for data
needs to occur.

The overhead of implementing true push is developing a server to listen on
the device.

 

[im.djmatty]

Hi Simon

When you say develop a server on your device to listen for push
messages, I am interested to know how you get around the issues of IP
addressing, and the mobile networks use of NAT and PAT on their
standard mobile internet connections?

Or are you proposing that the device uses a private APN or VPN
connection and as such each device has a unique IP address and can be
addressed from another machine that wants to talk to the listening
process on the device?

The "direct push" technology that is implemented in MSFT and Exchange
SP2 gets around a lot of these issues by having the connection
initiated by the device, and just waiting for a response.

I'd be interested to know your thoughts on addressing these issues

Thanks

Matt

 

[Scott Gifford]

Hi Scott,


You are right about this if you haven't implemented true push technology.
However, if you have implemented true push technology, ie: a server process
listens on the device waiting for the back office to connect to it, once
connected, pushs out the message then disconnects, then you will remove the
need for polling.

Hi Simon,

I wondered the same things as Matt. When my device connects to the
GPRS network, the IP address I get is in RFC 1918 space (10.* to be
specific). How would my back office application contact my device
directly if the device were running a server?

When I connect from my device to the back office server, the
connection comes from a routable address, but if I don't send data
every few minutes the connection stops working, which I assume is my
carrier timing out my NAT connection. Given that, even if my server
had the routable IP address for my device, I doubt I could get packets
to it directly.

Also, I noticed the device's routable IP address will change
throughout the day, so it would need some way of finding that out, and
notifying the server of its new address.

Have you had much luck implementing true push that deals with these
issues on the wild Internet and the even wilder carrier networks?

Thanks!

---Scott.

 

[Guest]

Hi, Matt, Scott,

When you say develop a server on your device to listen for push
messages, I am interested to know how you get around the issues of IP
addressing, and the mobile networks use of NAT and PAT on their
standard mobile internet connections?

This is an interesting topic and very real world. My client has a private
APN which works via a leased line between the GGSN and the CPN (Converged
Packet Network)to our clients LAN, so when our devices connect to the private
APN, the GGSN then releases a dynamic IP address from a pool of
pre-configured reusable addresses. The GGSN talks directly to the CPN in fact
it bridges the gap between the GPRS network and the CPN. The CPN then
connects via a leased line channel to the corporate LAN. You can use IPSec
here instead or use a combination of both (failover).

Using this infrastructure allows us to have a two communication from and to
the devices in the field from the back office running on the LAN.

Our client chose to use dynamic IP addresses because static addresses per
SIM are very expensive. Of course this lead to an interesting problem - how
do we track the IP address. The solution is simple, the device monitors the
GPRS network adapter and when the IP address changes the device sends a
message to one of our back end gateway servers (any one of 16 scalled
servers) running on the LAN. The only overhead here is the size of the
message that needs to be sent on IP address change. But we find in reality
the IP's don't change that often.

Or are you proposing that the device uses a private APN or VPN
connection and as such each device has a unique IP address and can be
addressed from another machine that wants to talk to the listening
process on the device?

Yes. As you know using a public APN won't work because of the NAT(Network
Address Translation) that has to happen. As you mentioned we have had working
a connection to a public APN (doesn't matter which) then connect via a VPN.
This works but is not very reliable. Your code will need to be very robust
for this.

The "direct push" technology that is implemented in MSFT and Exchange
SP2 gets around a lot of these issues by having the connection
initiated by the device, and just waiting for a response.

This solution also uses ActiveSync. We've not really looked into the direct
push via exchange in great detail, but I don't like the look of it.

Simon.

Matt

 

[Grzegorz Makarewicz]

Hi, Matt, Scott,


This is an interesting topic and very real world. My client has a private
APN which works via a leased line between the GGSN and the CPN (Converged
Packet Network)to our clients LAN, so when our devices connect to the private
APN, the GGSN then releases a dynamic IP address from a pool of
pre-configured reusable addresses. The GGSN talks directly to the CPN in fact
it bridges the gap between the GPRS network and the CPN. The CPN then
connects via a leased line channel to the corporate LAN. You can use IPSec
here instead or use a combination of both (failover).

Using this infrastructure allows us to have a two communication from and to
the devices in the field from the back office running on the LAN.

U'r right )
Consider using better devices, more safe, more production avare.
Not access point dependen.
See'u at nearest Motorola/Symbol conference.


MS - is fun for kiddies (in this task) - not so reliable, not so
satble - not so usefull


mak

 

[Guest]

Microsoft has nothing to do with the inability to route to a mobile device.
That's inherent in how the carriers operate. You'd have the same problem
under Palm, RIM, Linux, VxWorks or a cheese OS running through a macaroni
processor.

Not sure why it is that everyone loves to blame Microsoft for any technology
problems they run across. The power goes out a lot at my house - must be
Microsoft's unstable platform causing the trees to fall into the lines.


--

Chris Tacke, eMVP
Join the Embedded Developer Community
http://community.opennetcf.com

 

[im.djmatty]

Hi Simon

Thanks for the info, our push mechanism uses the same idea as yours,
in that it will only work when the network is configured to provide
the devices with routable ip addresses. I did start to look at using
STUN or some other NAT traversal method to get the packets to the
device, but haven't got too far with that at present.

I did some research into AUTD and how it works, it seems that it makes
an HTTP request to the server with a long timeout (the default i think
is 20 mins), and if anything on the exchange server changes for that
user the request returns with a response containing which mail folders
have changed, then activesync is kicked off and syncs only those
folders... if the request times out the device sends another request.
The push mechanism doesn't actually need to use activesync, it's just
a notification that the device needs to do a sync.

It's an interesting concept as it does mean that the devices don't
need any of that expensive network configuration, e.g. leased lines,
ipsec tunnels, private ip address pools etc, and it does work over the
public apns.

Matt

 

[im.djmatty]

Sorry, this hasn't really got anything to do with the devices used or
the operating systems on those devices, it's all specific to the
mobile network operator.

I know of european network operators that do provide unique public ip
addresses to each device on their network (or at least used to), but
until we get IPv6 NAT and PAT are going to be the norm for public APNs
of the networks...

Please tell me how symbol/motorola propose to allow me to use their
device and uniquely address it when it's connected over O2s mobile
GPRS network, or Vodafones, or Verizons?

Matt

 

[im.djmatty]

Well said Chris

Microsoft has nothing to do with the inability to route to a mobile device.
That's inherent in how the carriers operate. You'd have the same problem
under Palm, RIM, Linux, VxWorks or a cheese OS running through a macaroni
processor.

Not sure why it is that everyone loves to blame Microsoft for any technology
problems they run across. The power goes out a lot at my house - must be
Microsoft's unstable platform causing the trees to fall into the lines.

 

[Guest]

You can use a public APN via a VPN for a cheaper alternative to a leased line
or IPSec. But of course I wouldn't recomend it as public APN's are not that
reliable for mission-critical LOB apps. Public APN's are only meant for
internet browsing or email access etc.
--
Simon Hart
Visual Developer - Device Application Development MVP
http://simonrhart.blogspot.com

Hi Simon

Thanks for the info, our push mechanism uses the same idea as yours,
in that it will only work when the network is configured to provide
the devices with routable ip addresses. I did start to look at using
STUN or some other NAT traversal method to get the packets to the
device, but haven't got too far with that at present.

I did some research into AUTD and how it works, it seems that it makes
an HTTP request to the server with a long timeout (the default i think
is 20 mins), and if anything on the exchange server changes for that
user the request returns with a response containing which mail folders
have changed, then activesync is kicked off and syncs only those
folders... if the request times out the device sends another request.
The push mechanism doesn't actually need to use activesync, it's just
a notification that the device needs to do a sync.

It's an interesting concept as it does mean that the devices don't
need any of that expensive network configuration, e.g. leased lines,
ipsec tunnels, private ip address pools etc, and it does work over the
public apns.

Matt

Hi, Matt, Scott,

When you say develop a server on your device to listen for push
messages, I am interested to know how you get around the issues of IP
addressing, and the mobile networks use of NAT and PAT on their
standard mobile internet connections?

This is an interesting topic and very real world. My client has a private
APN which works via a leased line between the GGSN and the CPN (Converged
Packet Network)to our clients LAN, so when our devices connect to the private
APN, the GGSN then releases a dynamic IP address from a pool of
pre-configured reusable addresses. The GGSN talks directly to the CPN in fact
it bridges the gap between the GPRS network and the CPN. The CPN then
connects via a leased line channel to the corporate LAN. You can use IPSec
here instead or use a combination of both (failover).

Using this infrastructure allows us to have a two communication from and to
the devices in the field from the back office running on the LAN.

Our client chose to use dynamic IP addresses because static addresses per
SIM are very expensive. Of course this lead to an interesting problem - how
do we track the IP address. The solution is simple, the device monitors the
GPRS network adapter and when the IP address changes the device sends a
message to one of our back end gateway servers (any one of 16 scalled
servers) running on the LAN. The only overhead here is the size of the
message that needs to be sent on IP address change. But we find in reality
the IP's don't change that often.

Or are you proposing that the device uses a private APN or VPN
connection and as such each device has a unique IP address and can be
addressed from another machine that wants to talk to the listening
process on the device?

Yes. As you know using a public APN won't work because of the NAT(Network
Address Translation) that has to happen. As you mentioned we have had working
a connection to a public APN (doesn't matter which) then connect via a VPN.
This works but is not very reliable. Your code will need to be very robust
for this.

The "direct push" technology that is implemented in MSFT and Exchange
SP2 gets around a lot of these issues by having the connection
initiated by the device, and just waiting for a response.

This solution also uses ActiveSync. We've not really looked into the direct
push via exchange in great detail, but I don't like the look of it.

Simon.

Matt

Hi Scott,

It seems that connecting when required, sending the data, then
disconnecting is ideal for when the client is generating data and
sending it to the server.

However, if the server is generating data and sending it to the client
(for example push email), it seems to me that this approach basically
becomes polling the server, and it's much harder to know when to
connect. In particular, if your goal is to receive messages from the
server as quickly as possible, you're stuck with frequent polling
(which uses more power), or longer delays.

You are right about this if you haven't implemented true push technology.
However, if you have implemented true push technology, ie: a server process
listens on the device waiting for the back office to connect to it, once
connected, pushs out the message then disconnects, then you will remove the
need for polling.

AUTD takes advantage of the GPRS radio waking up the device, and so it
can basically initiate the connection, then go into suspend mode until
and unless notifications are available.

This is true in a true push type environment as well. Of course you would
need to connect to your APN via GPRS/UMTS when your application starts so
that the back office can see the device.

Of course, reality is more complicated (for example keepalive messages
must be sent to keep the connection up), but I think the basic
tradeoffs remain.

This will be a much more expensive approach to using true push technology.

What are your thoughts on a system in which a Windows Mobile client
must receive messages from a server as quickly as possible?

As I said above, you will always need a GPRS connection to your APN, but
with true push technology the only data going up and down the wire is the
data you want to send receive. No polling or keep-alives or checking for data
needs to occur.

The overhead of implementing true push is developing a server to listen on
the device.

 

[Scott Gifford]

You can use a public APN via a VPN for a cheaper alternative to a
leased line or IPSec.

But for a single application, is there any advantage to keeping a VPN
connection up all the time to handle incoming packets, rather than
keeping a TCP connection up all the time to handle incoming packets?

Both will require occasional keepalive packets I assume, to avoid NAT
timeouts by the providers. Other than that, both will only send
packets for startup and shutdown.

-----Scott.

 

[im.djmatty]

If you have a VPN connection up then you don't need to maintain a TCP
connection and you can have a socket listening on the device side as
it can\will have a routable ip address from the LAN.This will allow an
application on the LAN to initiate a socket connection to the device
if it knows the address when it needs to push data to the device.

If you don't use VPN then you will need to either maintain a TCP
connection with the server initiated from the device, or use some form
of NAT traversal to obtain an address and port that the device is
listening on. (I'm not sure if this is possible, I did start to look
at using a STUN client on my device to see if it was possible but
haven't had time to get very far with it)

Matt