General Issue: I need to create some data classes/structs that can be
used efficiently with both limited memory use as well as good search
capabilities.
Details: I'm working on a C# VSTO add-in, and part of its tasks is to
match dimensions between OLAP cubes. I'm collecting all the dimension
members as List<CubeInfo>, and then adding that List<CubeInfo> to
CubeDimensions. The end result will need to match the members of the
lists and return them in a different structure. It works fine from a
functional perspective, but it eats memory, since the members can
number from 5 to 800,000, and there are approx. 20 dimensions.
Question: What data structures can I employ that would have a limited
memory footprint, as well as provide good search capabilities on
millions of 'rows'
Additional: I've considered hashtables, as well as storing the
information in a SQL Db, since that will provide native matching
capabilities.
Existing Data structures:
[Serializable]
public class CubeInfo
{
public string DimensionType;
public string UniqueName;
public string Name;
}
[Serializable]
public class CubeDimensions
{
public string DimenstionName;
public List<CubeInfo> DimenstionData = new List<CubeInfo>
{ };
}
Arne Vajhøj View profile
More options Jul 27, 11:08 am
Newsgroups: microsoft.public.dotnet.languages.csharp
From: Arne Vajhøj <
[email protected]>
Date: Tue, 27 Jul 2010 11:08:58 -0400
Local: Tues, Jul 27 2010 11:08 am
Subject: Re: List/Collection/Array Selection - Memory Issues
Reply | Reply to author | Forward | Print | Individual message | Show
original | Report this message | Find messages by this author
On 27-07-2010 11:01, james wrote:
- Hide quoted text -
- Show quoted text -
General Issue: I need to create some data classes/structs that can be
used efficiently with both limited memory use as well as good search
capabilities.
Details: I'm working on a C# VSTO add-in, and part of its tasks is to
match dimensions betyween OLAP cubes. I'm collecting all the
dimension members as List<CubeInfo>, and then adding that
List<CubeInfo> to CubeDimensions. The end result will need to match
the members of the lists and return them in a different structure. It
works fine from a functional perspective, but it eats memory, since
the members can number from 5 to 800,000, and there are approx. 20
dimesions.
Question: What data structures can I employ that would have a limited
memory footprint, as well as provide good search capabilities on
millions of 'rows'
Additonal: I've considered hashtables, as well as storing the
information in a SQL Db, since that will provide native matching
capabilities.
Exiting Data structures:
[Serializable]
public class CubeInfo
{
public string DimensionType;
public string UniqueName;
public string Name;
}
[Serializable]
public class CubeDimensions
{
public string DimenstionName;
public List<CubeInfo> DimenstionData = new List<CubeInfo> { };
}
Maybe you can use Dictionary<int,CubeInfo> instead of
List<CubeInfo> !?
Arne
Reply Reply to author Forward Report spam
Reporting spam
Message reported
Rate this post: Text for clearing space
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before
posting.
You do not have the permission required to post.
james View profile
More options Jul 27, 11:16 am
Newsgroups: microsoft.public.dotnet.languages.csharp
From: james <
[email protected]>
Date: Tue, 27 Jul 2010 08:16:57 -0700 (PDT)
Local: Tues, Jul 27 2010 11:16 am
Subject: Re: List/Collection/Array Selection - Memory Issues
Reply | Reply to author | Forward | Print | Individual message | Show
original | Remove | Report this message | Find messages by this
author
- Hide quoted text -
- Show quoted text -
On 27-07-2010 11:01, james wrote:
General Issue: I need to create some data classes/structs that can be
used efficiently with both limited memory use as well as good search
capabilities.
Details: I'm working on a C# VSTO add-in, and part of its tasks is to
match dimensions betyween OLAP cubes. I'm collecting all the
dimension members as List<CubeInfo>, and then adding that
List<CubeInfo> to CubeDimensions. The end result will need to match
the members of the lists and return them in a different structure. It
works fine from a functional perspective, but it eats memory, since
the members can number from 5 to 800,000, and there are approx. 20
dimesions.
Question: What data structures can I employ that would have a limited
memory footprint, as well as provide good search capabilities on
millions of 'rows'
Additonal: I've considered hashtables, as well as storing the
information in a SQL Db, since that will provide native matching
capabilities.
[Serializable]
public class CubeInfo
{
public string DimensionType;
public string UniqueName;
public string Name;
}
[Serializable]
public class CubeDimensions
{
public string DimenstionName;
public List<CubeInfo> DimenstionData = new List<CubeInfo> { };
}
Maybe you can use Dictionary<int,CubeInfo> instead of
List<CubeInfo> !?
Arne- Hide quoted text -
- Show quoted text -
I need the first column as a key, since that will hold the dimension
name.
