How to dispose of array of struc that is reused many times

[Jim]

In a C# project I'm working on for an iterative design application, I need
to dispose of a large arrray of a struct object and reinitialize the
array between iterations.

That is, the user starts a design, and the program creates an array of
objects defined as follows:

public struct DES_TYPE
{
public string short_string;
public string long_string;
public int entryNo;
public int col;
public int row;
};

The class contains an array of the struct above and two variables that
specify the size of the array:

public class PrincipalClass
{

// create reference to array
public DES_TYPE [,] desType;

// array size
int noCols, noRows;



// class constructor
public PrincipalClass ( int no_cols, int no_rows)
{

// initialize variables
noCols = no_cols;
noRows = no_rows;

// allocate array
desType = new DES_TYPE [ noCols, noRows];

// code to initialize array elements

} // end class


Each time the user starts a new design, the array should be initialized.
It is not known beforehand what its size will be, although a typical
size is on the order of a million objects. I need some
way of freeing the memory. I can't quite see a way of
implementing the IDisposable interface here. The
mechanics of the creating the Dispose ( ) method
aren't in question. What I can't see is how to
dispose of the elements in the struct, that is,
the two strings and three integers *and reclaiming the
memory.*

Anybody see a way around this? For one thing, I don't
mind creating a single instance of the PrincipalClass
when the program is started, and then intializing the array
for each design iteration. But I need a way to free up the
memory used by the array before starting a new design iteration.

What am I overlooking?

 

[Barry Kelly]

In a C# project I'm working on for an iterative design application, I need
to dispose of a large arrray of a struct object and reinitialize the
array between iterations.

// allocate array
desType = new DES_TYPE [ noCols, noRows];

Each time the user starts a new design, the array should be initialized.
It is not known beforehand what its size will be, although a typical
size is on the order of a million objects. I need some
way of freeing the memory.

You need to pool your arrays, and to get reusable pools, you probably
need to stay away from two-dimensional arrays and do the column & row
multiplication & addition yourself. That way, you can allocate arrays
that are larger than needed / use arrays that are larger than strictly
needed to improve reusability of allocated arrays.

Basically, the solution to your problem is called memory pooling - it's
a well-known technique even outside of GC languages. There should be
lots of examples on Google.

I can't quite see a way of
implementing the IDisposable interface here.

IDisposable is not for memory.

-- Barry

 

[rossum]

In a C# project I'm working on for an iterative design application, I need
to dispose of a large arrray of a struct object and reinitialize the
array between iterations.

That is, the user starts a design, and the program creates an array of
objects defined as follows:

public struct DES_TYPE
{
public string short_string;
public string long_string;
public int entryNo;
public int col;
public int row;
};

The class contains an array of the struct above and two variables that
specify the size of the array:

public class PrincipalClass
{

// create reference to array
public DES_TYPE [,] desType;

// array size
int noCols, noRows;



// class constructor
public PrincipalClass ( int no_cols, int no_rows)
{

// initialize variables
noCols = no_cols;
noRows = no_rows;

// allocate array
desType = new DES_TYPE [ noCols, noRows];

// code to initialize array elements

} // end class


Each time the user starts a new design, the array should be initialized.
It is not known beforehand what its size will be, although a typical
size is on the order of a million objects. I need some
way of freeing the memory. I can't quite see a way of
implementing the IDisposable interface here. The
mechanics of the creating the Dispose ( ) method
aren't in question. What I can't see is how to
dispose of the elements in the struct, that is,
the two strings and three integers *and reclaiming the
memory.*

Anybody see a way around this? For one thing, I don't
mind creating a single instance of the PrincipalClass
when the program is started, and then intializing the array
for each design iteration. But I need a way to free up the
memory used by the array before starting a new design iteration.

What am I overlooking?

Set all references to the array to null and call System.GC.Collect()
which will force the garbage collector to run. This may take some
time if it has a lot of work to do so your users might notice a pause.

rossum

 

[Barry Kelly]

Set all references to the array to null and call System.GC.Collect()
which will force the garbage collector to run. This may take some
time if it has a lot of work to do so your users might notice a pause.

This is using a sledgehammer to pound in a nail, IMHO. The cost of using
large arrays without pooling is that gen 2 collections occur
occasionally. Calling GC.Collect() will force a gen 2 collection every
time you call it.

-- Barry

 

[Jim]

This is using a sledgehammer to pound in a nail, IMHO. The cost of using
large arrays without pooling is that gen 2 collections occur
occasionally. Calling GC.Collect() will force a gen 2 collection every
time you call it.

-- Barry

Barry and rossum:

Thanks to *both of you* for good suggestions. I may be forced to go the
memory pool route, but it does violate a ground rule here: I would
rather the user's design dictate the resources consumed (there are many
other objects besides the array I described earlier), rather than
having the resources dictate one facet of the largest design.
In other words, if I use a technique that cleans up between
iterations, then the user's iteration can trade off maximum
array size for maximum print buffer size which is only
loosely coupled to the array size. The print buffer is
built as the design iteration progresses; each step goes
practically unnoticed by the user. But building a print buffer
all at once... well, it would give him time to go fetch
a fresh cup of coffee.

It does so happen I globbed onto the GarbageCollector
Collect ( ) method after posting my earlier message here,
and found it acceptable up to about 500 k objects.
After that, yes, there is a bit of a balky delay although
shorter than the program's startup initialization which
reads a database. A finicky user may do a dozen
design iterations, and more, depending on how much
of a perfectionist s/he is.

I don't know which way I'm going to proceed just yet.
I'm going to fetch a cup of coffee and run some more
experiments. I'm tempted to offer both features
to the user and call them "Optimize for speed" or
"Optimize for memory usage". This has to be
traded off with how much effort we want to
spend explaining the ramifications to the
user who only knows what he sees on the
screen and doesn't care about the code
behind it all.

 

[Barry Kelly]

Thanks to *both of you* for good suggestions. I may be forced to go the
memory pool route, but it does violate a ground rule here: I would
rather the user's design dictate the resources consumed (there are many
other objects besides the array I described earlier), rather than
having the resources dictate one facet of the largest design.

I have my own pooled array generic template, and I use a wrapper
PooledArraySlice<T> which implements IDisposable. When it's disposed, it
hands the buffer back to the pool. In that way, it's just like other
resources. I'm not sure if this is what you're talking about though, but
I thought I'd just mention it.

-- Barry

 

[Jim]

I have my own pooled array generic template, and I use a wrapper
PooledArraySlice<T> which implements IDisposable. When it's disposed, it
hands the buffer back to the pool. In that way, it's just like other
resources. I'm not sure if this is what you're talking about though, but
I thought I'd just mention it.

-- Barry

Hmm. Anything you'd care to share?

Jim

 

[Barry Kelly]

Hmm. Anything you'd care to share?

I'll post it here, but be aware that it's part of a larger library that
is under current development (it's based around ideas developed for a
past project), and it uses a helper class to create exceptions, not
included. Use it for educational value, no more I've also removed its
namespace etc.

-- Barry

 

[Barry Kelly]

Hmm. Anything you'd care to share?

Oh, and I just added that 'this[int index]' property, but it has a bug -
it uses '_index' rather than 'index'. I normally don't access the array
slice directly, but via other mechanisms. The array pool is a lowly
component in a greater scheme

-- Barry