DataSets in CF?

  • Thread starter Thread starter Neil Kiser
  • Start date Start date
N

Neil Kiser

I just read something on a web site that said I should not be using
DataSets on the COmpact Framework due to performance considerations?
Is this true? If so, what should I be using?

My app will, eventually, be holding a number of 'datasets' in memory
as it is a data intensive application.

Thanks,
-Neil K.
 
You can use DataSets on Compact Framework. A DataSet is an in-memory data
structure, so it's impractical to hold dataset's with thousands of records,
due to the limited memory available on a mobile device as opposed to a
desktop machine. So you can use them with a little consideration as to their
size.

What is the source of your data (e.g. Sql Ce?), and what sort of volume of
data do you expect to be processing?

Peter

--
Peter Foot
Windows Embedded MVP
www.inthehand.com | www.opennetcf.org

Do have an opinion on the effectiveness of Microsoft Windows Mobile and
Embedded newsgroups? Let us know!
https://www.windowsembeddedeval.com/community/newsgroups
 
I am currently using SQL CE for my datastore. I will have, in one
case anyway, over a thousand records, likely more. Many of my tables
are small 25-60 records, but some will be large.

Thanks,
-Neil K.
 
In which case you should try to work directly with the database using
DataReader where possible. When you do need to fill a DataSet e.g. this is
convenient for binding to forms controls, then you should try and restrict
the number of records you work with at any particular time using a select
query when filling the DataSet.

Peter

--
Peter Foot
Windows Embedded MVP
www.inthehand.com | www.opennetcf.org

Do have an opinion on the effectiveness of Microsoft Windows Mobile and
Embedded newsgroups? Let us know!
https://www.windowsembeddedeval.com/community/newsgroups
 
Yes, I have been concerned about my data access strategy, but I had
not progressed to this point in my design stage yet.

I do have a little bit of confusion on this subject of DataSet vs
DataReader. Here's what I think I know:

- With a DataSet I populate it, it's in memory, and I perform a
databind with a control. Works well, but I'm uncertain just how many
times the data structure is duplicated.
- With a DataReader, based on what i have read since reading your
e-mail yesterday, I create it and use it to populate a control, and
then I close the DataReader. It would seem clear that I'm only
getting one copy of the data in this scenario.

My question is this, is there no mechanism in which the control can
have a an open connection to the database, such that the control
(let's say a list view) can only have in memory the records that it
needs to display? In other words, if there is a 1000 row table that a
list view is going to allow the user to view, that only the 10 or so
that are currently in view would be loaded into memory? I ask this
because, as I went to bone-up on the DataReader this is what I was
expecting to find. Instead, I found something that would seem to be
no better than a DataSet provided that the data in the dataset wasn't
being duplicated.

DataReader = 1 loading of 1000-row table data

DataSet = 1 loading of 1000-row table data *but, when I set a control
to databind, am I in effect duplicating the data structure?
 
If you need to show data in the grid (or other bindable control), use
DataSet/DataAdapter, not DataReader.
Data from DataSet is not duplicated in the control if you use Data Binding
and it is faster than populating the control with data from DataReader.
You also can sort and filter data in DataSet without executing another
query.

Please see this for performance data on that:

http://groups.google.com/groups?hl=en&lr=&selm=cfSAqAZIEHA.4060@cpmsftngxa
06.phx.gbl&rnum=7

Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
From: Neil Kiser <neil@SPAM_GUARD_REMOVE_TESTkiservilletest.com>
Subject: Re: DataSets in CF?
Date: Sat, 09 Oct 2004 05:34:34 -0500
Message-ID: <[email protected]>
References: <[email protected]>
Click to expand...
<#[email protected]>
 
But the message I have been getting is that for recordsets as large as
the one I will be working with that, on a PDA, I should not be using a
DataSet. I'm rather confused now :(
 
DataReader is really an option for processing data sequentially, if you are
databinding controls you should use a DataSet but you should consider
whether you can filter the data down first - are you sure you need 1000+
records accessible all the time?

Peter

--
Peter Foot
Windows Embedded MVP
www.inthehand.com | www.opennetcf.org

Do have an opinion on the effectiveness of Microsoft Windows Mobile and
Embedded newsgroups? Let us know!
https://www.windowsembeddedeval.com/community/newsgroups
 
I only have a single scenario in which I would have a 1000+ items in a
ListView. The scenario is very much like the one you face when you
have a list of contacts in a contacts database. The user expects to
scroll down past the A's and into the B's, C's, etc.

One thing I am planning to do is to keep the data in each row to a
minimum. Other than that, I'm not sure how I would restrict someone
from loading up fewer records.
 
That's the right message if you _can_ get away without storing all records
in memory.
In this case all you can do to avoid storing data in memory is to redesign
your application to eliminate the need for that.
If you can not, you have little choice. And using DataReader to populate
listview is way worse than using DataSet.

CF V2 has a new class called SqlCeResultSet which is a bindable live cursor
into SQL CE database.
With this class it does not matter how many records you have, it will only
load visible records (all 10 or so of them).
It will load other rows as you scroll records in the grid.

By the way, 1000+ records - it's not that much (unless this plus is like
5000).
It will take up to 10 seconds to load 1000 very complex records, after that
it's fast.

Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
From: Neil Kiser <neil@SPAM_GUARD_REMOVE_TESTkiservilletest.com>
Subject: Re: DataSets in CF?
Date: Sat, 09 Oct 2004 15:10:06 -0500
Message-ID: <[email protected]>
References: <[email protected]>
Click to expand...
<#[email protected]>
<[email protected]>
<[email protected]>
 
Neil,

Loading 1000+ records is going to be very slow on a pda.

I have a number of large tables - about 15000 rows in a drugs database and
the users do expect to be able scroll through them. But that is not
practical, so you can create a cache of records and read about 30 at a time.
Plus you can give them the ability to seek on a particular drug, so the more
letters they type the closer they get and the query naturally returns less
rows. So its really an illusion that they can scroll through the whole
table. Usually you only need to type 3 or so characters and you can then see
the drug required in the grid somewhere in the page of the grid. Tap on it
and you have all it details.

I don't bother with datasets usually, because of the extra overhead and just
use a DataTable.

Graham
 
But is it the number of records or is it the volume if data or the
complexity of the joins?

I do have 1000 records, which would likely grow by 100 record per
year. The data is not complex, but I do have to perform a join
between ~5 tables, to pull a single piece of data from each of the
secondary tables.

I exported the current DataSet to XML and removed all tags. The net
result is 297Kb worth of data, and I believe I will be able to cut the
data down by half. The number of rows would remain the same, it's
just that I am currently bringing back everything and I know I don't
need all of this data.

But I don't know about limiting it any further. I'm definitely
thinking about how I can limit the number of rows. I just know it's
going to be a pain in the keister. So at the moment, I'm working to
ensure that I fully understand the nature of the problem. Is it
memory? Is it performance? Can I compensate in other ways? etc.
etc. Once I'm sure I understand the problem and determine that I must
come up with an alternate solution, I would reengineer.

At this point I'm just not clear on what my problem is. First person
mentioned memory, now I'm hearing performance (on the dataset load).
So what I'm doing now is juts making sure that those-in-the-know
understand that it is 297Kb worth of data max today and 1,000 rows max
today. I expect that it would grow at 100 records per year and
probably 30Kb per year.

If, for a moment, we were to say that I was "ok" today, then I wonder
about the upcoming versions of .Net CF and SQL Server CE and the fact
that I could release an update in the coming years that would be in
place before someone accumulated enough data to move the app from
being "acceptable" to truly bad performance.

Thanks for your insights.
 
I would not worry much about 300K of data on 32+ Mb device.
Performance (of the first load) might be a limitation.
I would suggest doing a quick prototype to find out if performance is
acceptable to you.

Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
From: Neil Kiser <neil@SPAM_GUARD_REMOVE_TESTkiservilletest.com>
Subject: Re: DataSets in CF?
Date: Mon, 11 Oct 2004 05:48:38 -0500
Message-ID: <[email protected]>
References: <[email protected]>
Click to expand...
<#[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<#[email protected]>
 
Back
Top