array size. out of memory exception.

[Joel]

I'm sorry if this question has been asked before, but is it possible
to define a 2 D array of type double with dimensions of the order
65536 x 65536 ?
I'm working on a face recognition algorithm called PCA (Principal
component analysis), and at an intermediate stage, I need to work with
arrays of the above dimension.

Is it possible? I get a System out of memory exception for 65536x65536
And for arrays of size 10000 x 10000, the program runs really slow due
to virtual memory demands. Is there a work around for the problem?

Thanks a lot

Joel

 

[Willy Denoyette [MVP]]

I'm sorry if this question has been asked before, but is it possible
to define a 2 D array of type double with dimensions of the order
65536 x 65536 ?
I'm working on a face recognition algorithm called PCA (Principal
component analysis), and at an intermediate stage, I need to work with
arrays of the above dimension.

Is it possible? I get a System out of memory exception for 65536x65536
And for arrays of size 10000 x 10000, the program runs really slow due
to virtual memory demands. Is there a work around for the problem?

Thanks a lot

Joel

 

[Willy Denoyette [MVP]]

I'm sorry if this question has been asked before, but is it possible
to define a 2 D array of type double with dimensions of the order
65536 x 65536 ?
I'm working on a face recognition algorithm called PCA (Principal
component analysis), and at an intermediate stage, I need to work with
arrays of the above dimension.

You can't have arrays larger than 2GB in .NET. (both 32 and 64 bit)

Is it possible? I get a System out of memory exception for 65536x65536
And for arrays of size 10000 x 10000, the program runs really slow due
to virtual memory demands. Is there a work around for the problem?

No, not using managed array's (see above), you could use unmanaged array's larger than 2GB,
but this would require 64 bit hardware with +32GB RAM and a 64 bit OS.
10000*10000 double arrays shouldn't be a problem on 32 bit systems having +1GB of RAM
preferably 2GB.

Willy.

 

[pfc_sadr]

would it really be impossible to store this in a database?

 

[=?ISO-8859-1?Q?G=F6ran_Andersson?=]

I'm sorry if this question has been asked before, but is it possible
to define a 2 D array of type double with dimensions of the order
65536 x 65536 ?
I'm working on a face recognition algorithm called PCA (Principal
component analysis), and at an intermediate stage, I need to work with
arrays of the above dimension.

Is it possible? I get a System out of memory exception for 65536x65536
And for arrays of size 10000 x 10000, the program runs really slow due
to virtual memory demands. Is there a work around for the problem?

Thanks a lot

Joel

Do you really have to work with the entire image at once? I haven't done
any pattern recognition myself, but I would imagine that you would
always compare pixels that are close together.

Photoshop, for example, uses a swapfile to store the image data that is
not in memory. If you have a large enough image, it will never be
entirely in memory at once.