I'm sorry if this question has been asked before, but is it possible
to define a 2 D array of type double with dimensions of the order
65536 x 65536 ?
I'm working on a face recognition algorithm called PCA (Principal
component analysis), and at an intermediate stage, I need to work with
arrays of the above dimension.

Is it possible? I get a System out of memory exception for 65536x65536
And for arrays of size 10000 x 10000, the program runs really slow due
to virtual memory demands. Is there a work around for the problem?

Thanks a lot

Joel

"Joel" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> I'm sorry if this question has been asked before, but is it possible
> to define a 2 D array of type double with dimensions of the order
> 65536 x 65536 ?
> I'm working on a face recognition algorithm called PCA (Principal
> component analysis), and at an intermediate stage, I need to work with
> arrays of the above dimension.
>
> Is it possible? I get a System out of memory exception for 65536x65536
> And for arrays of size 10000 x 10000, the program runs really slow due
> to virtual memory demands. Is there a work around for the problem?
>
> Thanks a lot
>
> Joel
>

"Joel" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> I'm sorry if this question has been asked before, but is it possible
> to define a 2 D array of type double with dimensions of the order
> 65536 x 65536 ?
> I'm working on a face recognition algorithm called PCA (Principal
> component analysis), and at an intermediate stage, I need to work with
> arrays of the above dimension.
>
You can't have arrays larger than 2GB in .NET. (both 32 and 64 bit)

> Is it possible? I get a System out of memory exception for 65536x65536
> And for arrays of size 10000 x 10000, the program runs really slow due
> to virtual memory demands. Is there a work around for the problem?
>

No, not using managed array's (see above), you could use unmanaged array's larger than 2GB,
but this would require 64 bit hardware with +32GB RAM and a 64 bit OS.
10000*10000 double arrays shouldn't be a problem on 32 bit systems having +1GB of RAM
preferably 2GB.

Willy.

would it really be impossible to store this in a database?




On Feb 15, 9:48 am, "Willy Denoyette [MVP]"
<willy.denoye...@telenet.be> wrote:
> "Joel" <joelag...@gmail.com> wrote in message
>
> news:(E-Mail Removed)...> I'm sorry if this question has been asked before, but is it possible
> > to define a 2 D array of type double with dimensions of the order
> > 65536 x 65536 ?
> > I'm working on a face recognition algorithm called PCA (Principal
> > component analysis), and at an intermediate stage, I need to work with
> > arrays of the above dimension.
>
> You can't have arrays larger than 2GB in .NET. (both 32 and 64 bit)
>
> > Is it possible? I get a System out of memory exception for 65536x65536
> > And for arrays of size 10000 x 10000, the program runs really slow due
> > to virtual memory demands. Is there a work around for the problem?
>
> No, not using managed array's (see above), you could use unmanaged array's larger than 2GB,
> but this would require 64 bit hardware with +32GB RAM and a 64 bit OS.
> 10000*10000 double arrays shouldn't be a problem on 32 bit systems having +1GB of RAM
> preferably 2GB.
>
> Willy.

Joel wrote:
> I'm sorry if this question has been asked before, but is it possible
> to define a 2 D array of type double with dimensions of the order
> 65536 x 65536 ?
> I'm working on a face recognition algorithm called PCA (Principal
> component analysis), and at an intermediate stage, I need to work with
> arrays of the above dimension.
>
> Is it possible? I get a System out of memory exception for 65536x65536
> And for arrays of size 10000 x 10000, the program runs really slow due
> to virtual memory demands. Is there a work around for the problem?
>
> Thanks a lot
>
> Joel
>

Do you really have to work with the entire image at once? I haven't done
any pattern recognition myself, but I would imagine that you would
always compare pixels that are close together.

Photoshop, for example, uses a swapfile to store the image data that is
not in memory. If you have a large enough image, it will never be
entirely in memory at once.

--
Göran Andersson
_____
http://www.guffa.com