Online, Offline, Disconnected applications.

[Aaron A. Anderson]

I'm in the initial stages of architecting an application for use in both an
online and offline modes. I've looked at both the TaskVision application
and the Offline Application Block for for guidance.

The TaskVision application is much simpler in design than the Offline
Application Block, but the latter offers a much more robust solution. The
problem I have with the OAP is it's choices of offline data stores and how
the data gets synchronized. I don't want to have to force my customers to
install MSDE (getting them to install the .NET framework is hard enough),
but the addition of MSMQ makes it undoable proposition.

The options I see for effective offline data cache is:

1. DataSets saved as xml. My main concern here is the scalability,
efficiency, and memory usage when the application is running. Since
DataSets are essentially in-memory databases, I can see memory consumption
getting out of control after a while when the amount of data increases. On
the plus side, it simplifies the client-side data layer.

2. MSDE. With MSDE I get both increased memory use, plus I get to duplicate
the datalayer in both the web service and client. Can use replication, but
since this app is supposed to work over the Internet with Web Services, that
option is closed to me (I think).

3. MDBs. Poo poo them, but at least they're memory efficient, and can be
queried. The downside is having to create two separate datalayer
implementations. One for SQL, one for Access. The synchronization code is
a bit more complex than just using DataSets.

Has anyone else scratched their heads over the best route to take or am I
suffering Analysis Paralysis?

Can the experts out there share some wisdom?

 

[Val Mazur]

Hi Aaron,

1. Using DataSets will work, but you could be in a trouble in case if you
get pretty big XML file. In this case your downloading time (loading XML
into actual DataSet) could be pretty big. I do not know how much data you
are going to keep in a DataSet, but you could make some tests to see if it
is OK for you or not. Since you cannot query and open just part of the saved
XML file, this will be a disadvantage

2. Using MSDE (like for me) is the best way. First of all this the same SQL
Server database and provides you with almost all the features, which SQL
Server has. It would allow you to select only data, which you really need
and could improve performance of application. I do not think you should
worry about memory in this case, since you could select only small amount of
data and rest of it will be in a database. You could provide synchronization
as well. See next Kb about how to replicate MSDE

http://support.microsoft.com/default.aspx?scid=kb;en-us;324992&Product=sql

3. I do not think Access is a best idea. From one side it is just a simple
database file, but from other side it is less stable and requires repair one
in a while. Plus it has less functionality than MSDE

 

[Anthony Williams]

1. DataSets saved as xml. My main concern here is the scalability,
efficiency, and memory usage when the application is running. Since
DataSets are essentially in-memory databases, I can see memory consumption
getting out of control after a while when the amount of data increases.
On
the plus side, it simplifies the client-side data layer.

You may or may not be talking about website development with ASP.NET here,
but we've used a scenario like this. A few months back, a website we've got
had lots of problems with the SQL Server it uses, connectivity issues and
other problems which made real-time reading of the data a problem. Since our
website is updated only once a day (at around 3am) there's no major problem
if occasionally we can't insert data.

So, we have a sub which fires on Application Start and creates an
Application object (for arguments' sake, we'll say we're populating
Application("WebsiteData") with a DataSet). As it does this, it uses the
WriteXML method of the DataSet to persist the data, as XML, to a location on
the host server. On subsequent Starts, the Application will attempt to read
data from the SQL server, falling back to the persisted XML file if it
fails.

All our information retrieval subs/functions use a ByRef to
CType(Application("WebsiteData"), DataSet), and if/when the database is down
it doesn't matter, because the site does mainly read tasks, not write.
(Though all changes could be persisted back to the Application object, and
then persisted back to the database when the Application closes down.) Our
write subs/functions are a little more complex as they have to:

Lock the application
Write the data to the dataset
Persist changes to XML and SQL if available
Unlock the application

And of course, there's the issues with re-writing back to SQL when the app
shuts down.

I'm not sure whether or not this is the acceptable way to do it, but it
works for us. We're considering putting the same practices onto our
customer-facing website which already uses some of the methods discussed
above, but still reads/writes directly from/to SQL server.

Regards,


Anthony Williams
Internet Solutions Developer
Listers Group
www.listersgroup.co.uk

 

[Morten Mertner]

Hi Aaron,

I would recommend using SQLite as client-side database. It is free,
embeddable, very fast and impressively feature-complete. There is a
provider included with Mono that works well using ms.net (visit
www.sqlite.org for more info).

If you'd like to avoid writing all of your queries twice you can add an
abstraction layer like Gentle.NET (or any other persistence framework
supporting both databases). Gentle can be found at sf.net/projects/gopf.

It might also be relevant for you to study the Shadowfax project on
gotdotnet - it contains lots of code to simplify distributed development
(although I don't think database synchronization is part of the offering).

It should be possible for you to use XML DiffGrams to synchronize the
databases although it's a tricky problem, especially as you need to
synchronize changes both ways and possibly even handle conflicts.

Yours,
Morten