Generic object manipulation

[dsmith]

I have a class to store a data array and associated metadata, and the
array can be constructed either as (for illustrative purposes) float
or short. I would like to convert this to a generic version of the
class to avoid a lot of extra type checking throughout the class, and
speed up certain functions (testing using a fixed version of the
generic class showed a 20%-30% improvement in certain high-cost
functions).

So, the current version is:

public class DataArray
{
public DataArray(int rows, int cols, Type dataType)
{
switch (dataType)
case typeof(float):
localArray = new float[rows,cols];
break;
case typeof(short):
localArray = new short[rows,cols];
break;
}
}
}


And I would like to instead have:

public class DataArray<TArrayType>
{
public DataArray(int rows, int cols)
{
localArray = new TArrayType[rows, cols];
}
}


So far easy enough. However to allow data to be assigned from one
array to another I had to add a construction restriction:

public class DataArray<TArrayType> where TArrayType : IConvertible
{
public DataArray(int rows, int cols)
{
localArray = new TArrayType[rows, cols];
}

public TArrayType this[int row, int col]
{
get
{
return localArray[row, col];
}
set
{
localArray[row, col] = value;
}
}
}

I would have preferred to be able to specify the exact allowable
types, but I can filter further in the constructor. Unfortunately the
compiler gets touchy about it in various places.


So now I'd like to do some math. DataArray 1 (da1) and DataArray 2
(da2) can be of either float or short, and the result of some
calculation on the cells of each array will be put in DataArray 3
(da3); I know what type da3 is going to be, so that isn't a problem.

First I need to get da1 and da2 from the objects that are holding onto
them. And.... now I'm stuck. How to I define variables for generic
versions (ie: determined at run-time) of the DataArray? What does a
function that can return any type of DataArray look like?


Various attempts:

DataArray da1; // Using the generic type 'DataArray<TArrayType>'
requires '1' type arguments
DataArray<float> da1; // Well fine, it compiles, but now I'm locked
into using a specific type of DataArray, which is contrary to the
entire point of making it generic in the first place. Instead of type
checking inside the class as is done now, I just move the type
checking burden to outside the class.

Other error messages lead me to try
DataArray<IConvertible> da1; // It compiles!

DataArray<float> da2 = new DataArray<float>(1,2); // Assuming I can
get a function to return this, this is what we'll be assigning to da1.

da1 = da2 // Cannot implicitly convert type 'DataArray<float>' to
'DataArray<System.IConvertible>' --- ?? What? Ok, maybe an explicit
conversion...

da1 = (DataArray<IConvertible>)new DataArray<float>(1, 1); // Cannot
convert type 'DataArray<float>' to 'DataArray<System.IConvertible>'
--- Ok, not that way either.


And on top of this is trying to figure out a method signature that
will return a generic version of DataArray:

private DataArray<V> GetGenDA(int i)
{
if (i == 1)
return new GenericDataArray<float>();
else
return new GenericDataArray<short>();
}

// The type or namespace name 'V' could not be found (are you missing
a using directive or an assembly reference?)

And if I drop the <V> I get:

// Using the generic type 'DataArray<TArrayType>' requires '1' type
arguments


Another possibility: perhaps I can return the type of array contained
in each container object and use that to define the variables that are
going to accept the arrays.


private Type GetTypeOfDocument()
{
// example:
return typeof(DataArray<float>);
}

private void SomeOtherFunction()
{
Type docType = GetTypeOfDocument();
docType da1; // The type or namespace name 'docType' could not be
found (are you missing a using directive or an assembly reference?)
}


Or perhaps using reflection:

private Type GetDataArrayType()
{
return typeof(short);
}

private void MakeDAObject()
{
Type da1 = typeof(DataArray<>);
Type da1a = da1.MakeGenericType(GetDataArrayType());

// Fine up to here, where once again we can only create an instance
if we know the concrete type
// beforehand.
da1a da2 = (da1a)Activator.CreateInstance(da1a);
}



So in the end, nothing seems to work. Is what I'm trying to do at all
possible? It would seem to apply to any situation where you want to
consume/manipulate a generic object rather than perform work within
the scope that the generic was originally declared, or within the
generic object itself. Every time I think I've figured something out,
it just turns out that the problem got shifted one level up or down in
the code.

David

 

[Jon Skeet [C# MVP]]

I have a class to store a data array and associated metadata, and the
array can be constructed either as (for illustrative purposes) float
or short. I would like to convert this to a generic version of the
class to avoid a lot of extra type checking throughout the class, and
speed up certain functions (testing using a fixed version of the
generic class showed a 20%-30% improvement in certain high-cost
functions).

So, the current version is:

public class DataArray
{
public DataArray(int rows, int cols, Type dataType)
{
switch (dataType)
case typeof(float):
localArray = new float[rows,cols];
break;
case typeof(short):
localArray = new short[rows,cols];
break;
}
}
}

Where is localArray declared?

And I would like to instead have:

public class DataArray<TArrayType>
{
public DataArray(int rows, int cols)
{
localArray = new TArrayType[rows, cols];
}
}

That sounds reasonable.

So far easy enough. However to allow data to be assigned from one
array to another I had to add a construction restriction:

public class DataArray<TArrayType> where TArrayType : IConvertible
{
public DataArray(int rows, int cols)
{
localArray = new TArrayType[rows, cols];
}

public TArrayType this[int row, int col]
{
get
{
return localArray[row, col];
}
set
{
localArray[row, col] = value;
}
}
}

I would have preferred to be able to specify the exact allowable
types, but I can filter further in the constructor. Unfortunately the
compiler gets touchy about it in various places.

Yes, you can't just constrain types to be one of a particular set at
compile time - although you could throw an exception at execution time
(using typeof(TArrayType) to check the type involved).

So now I'd like to do some math. DataArray 1 (da1) and DataArray 2
(da2) can be of either float or short, and the result of some
calculation on the cells of each array will be put in DataArray 3
(da3); I know what type da3 is going to be, so that isn't a problem.

First I need to get da1 and da2 from the objects that are holding onto
them. And.... now I'm stuck. How to I define variables for generic
versions (ie: determined at run-time) of the DataArray? What does a
function that can return any type of DataArray look like?

Well, you could make the method generic:

DataArray<T> DoSomething<T> (...)

but I suspect that won't help you.

The main thing is that generics are primarily a way of making things
*more statically typed*. If you want the type to be determined at
execution time, you really need to start using reflection.

Various attempts:

DataArray da1; // Using the generic type 'DataArray<TArrayType>'
requires '1' type arguments
DataArray<float> da1; // Well fine, it compiles, but now I'm locked
into using a specific type of DataArray, which is contrary to the
entire point of making it generic in the first place. Instead of type
checking inside the class as is done now, I just move the type
checking burden to outside the class.

Other error messages lead me to try
DataArray<IConvertible> da1; // It compiles!

DataArray<float> da2 = new DataArray<float>(1,2); // Assuming I can
get a function to return this, this is what we'll be assigning to da1.

da1 = da2 // Cannot implicitly convert type 'DataArray<float>' to
'DataArray<System.IConvertible>' --- ?? What? Ok, maybe an explicit
conversion...

No, there's no explicit conversion. Generics aren't covariant by type
parameters.

da1 = (DataArray<IConvertible>)new DataArray<float>(1, 1); // Cannot
convert type 'DataArray<float>' to 'DataArray<System.IConvertible>'
--- Ok, not that way either.


And on top of this is trying to figure out a method signature that
will return a generic version of DataArray:

private DataArray<V> GetGenDA(int i)
{
if (i == 1)
return new GenericDataArray<float>();
else
return new GenericDataArray<short>();
}

You can't do that - how would the compiler be able to validate
anything?

So in the end, nothing seems to work. Is what I'm trying to do at all
possible? It would seem to apply to any situation where you want to
consume/manipulate a generic object rather than perform work within
the scope that the generic was originally declared, or within the
generic object itself. Every time I think I've figured something out,
it just turns out that the problem got shifted one level up or down in
the code.

Just keep bearing in mind that generics are primarily there to provide
more compile-time support. If you want to make a method which will
generically deal with data arrays of two different types, it will need
to be a generic method (with two type parameters, almost certainly).

However, it's not really terribly clear to me what you're really trying
to achieve to start with, unfortunately

 

[dsmith]

I have a class to store a data array and associated metadata, and the
array can be constructed either as (for illustrative purposes) float
or short. I would like to convert this to a generic version of the
class to avoid a lot of extra type checking throughout the class, and
speed up certain functions (testing using a fixed version of the
generic class showed a 20%-30% improvement in certain high-cost
functions).

So, the current version is:

public class DataArray
{
public DataArray(int rows, int cols, Type dataType)
{
switch (dataType)
case typeof(float):
localArray = new float[rows,cols];
break;
case typeof(short):
localArray = new short[rows,cols];
break;
}
}
}

Where is localArray declared?

It's a class member variable. Left it out as not really important for
the example. And I messed up writing that up. Should be a float[,]
localFloatArray and a short[,] localShortArray, respectively.

Well, you could make the method generic:

DataArray<T> DoSomething<T> (...)

but I suspect that won't help you.

The main thing is that generics are primarily a way of making things
*more statically typed*. If you want the type to be determined at
execution time, you really need to start using reflection.

Well, the problem is that even using reflection really didn't get me
very far, since I still need to know what the concrete type is before
I can assign it to a variable.

Just keep bearing in mind that generics are primarily there to provide
more compile-time support. If you want to make a method which will
generically deal with data arrays of two different types, it will need
to be a generic method (with two type parameters, almost certainly).

However, it's not really terribly clear to me what you're really trying
to achieve to start with, unfortunately

I need to be able to manipulate the data elements within the arrays,
and have built a number of functions to handle that. EG: difference
between ArrayA and ArrayB; linear regression applied to ArrayC between
ArrayA and ArrayB; noise removal on the data elements of ArrayA; etc.
It would be silly to put all these functions inside the DataArray
class itself, as they all need their own parameter settings and
various interface hooks, so they're set up in their own independant
classes.

However they still need to be able to operate on the DataArray
objects, and as such need to be able to have variables referencing the
DataArrays, and need to be able to request DataArrays from other
objects.


Gave some thought to the generic method, and tried out this framework:

internal class Testing
{
internal Type GetTypeA()
{
return typeof(float);
}

internal Type GetTypeB()
{
return typeof(ushort);
}

internal void CallFunc()
{
Process<GetTypeA(), GetTypeB()>();
}


internal DataArray<float> Process<T, U>()
where T : struct, IConvertible
where U : struct, IConvertible
{
DataArray<T> arr1 = new DataArray<T>(1,2);
DataArray<U> arr2 = new DataArray<U>(3,4);

// do stuff
return new DataArray<float>(5,6);
}
}


Looks like it would work pretty well, except for the call to Process()
in CallFunc, as I can't seem to find any way to pass the basic type
used to the generic T/U values. If I could just apply a typeof()
result to a generic constructor I think I could work things out.


David

 

[Jon Skeet [C# MVP]]

So, the current version is:

public class DataArray
{
public DataArray(int rows, int cols, Type dataType)
{
switch (dataType)
case typeof(float):
localArray = new float[rows,cols];
break;
case typeof(short):
localArray = new short[rows,cols];
break;
}
}
}

Where is localArray declared?

It's a class member variable. Left it out as not really important for
the example. And I messed up writing that up. Should be a float[,]
localFloatArray and a short[,] localShortArray, respectively.

So in your current version you've got two different types of array
variables?

Well, the problem is that even using reflection really didn't get me
very far, since I still need to know what the concrete type is before
I can assign it to a variable.

Well, you can avoid that by using just "object" of course, but it's not
very satisfactory.

I think this is the key to the whole problem though - working out how
much information you actually have at compile-time, and where.

I need to be able to manipulate the data elements within the arrays,
and have built a number of functions to handle that. EG: difference
between ArrayA and ArrayB; linear regression applied to ArrayC between
ArrayA and ArrayB; noise removal on the data elements of ArrayA; etc.
It would be silly to put all these functions inside the DataArray
class itself, as they all need their own parameter settings and
various interface hooks, so they're set up in their own independant
classes.

However they still need to be able to operate on the DataArray
objects, and as such need to be able to have variables referencing the
DataArrays, and need to be able to request DataArrays from other
objects.

That sounds like they should be generic methods then.

Gave some thought to the generic method, and tried out this framework:

internal class Testing
{
internal Type GetTypeA()
{
return typeof(float);
}

internal Type GetTypeB()
{
return typeof(ushort);
}

internal void CallFunc()
{
Process<GetTypeA(), GetTypeB()>();
}


internal DataArray<float> Process<T, U>()
where T : struct, IConvertible
where U : struct, IConvertible
{
DataArray<T> arr1 = new DataArray<T>(1,2);
DataArray<U> arr2 = new DataArray<U>(3,4);

// do stuff
return new DataArray<float>(5,6);
}
}

Looks like it would work pretty well, except for the call to Process()
in CallFunc, as I can't seem to find any way to pass the basic type
used to the generic T/U values. If I could just apply a typeof()
result to a generic constructor I think I could work things out.

Well again you *can* use reflection on generic methods, but it's a bit
painful. The point is that if you don't know at compile-time what the
type parameters are when you call Process, then generics aren't really
going to help you. Of course, if you're calling it from another generic
method you can use the type parameters again, but sooner or later
you're going to need compile-time information, or you'll be forced to
use reflection.

 

[dsmith]

So in your current version you've got two different types of array
variables?

Three, actually, but one is very rarely used. If I can get it working
with two, adding more shouldn't be a problem.

Well, you can avoid that by using just "object" of course, but it's not
very satisfactory.

I think this is the key to the whole problem though - working out how
much information you actually have at compile-time, and where.

At compile time I have no way of knowing what the precise input type
is going to be (other than it's going to be one of the numeric types
that DataArray will accept), though I should always know what the
output type will be (either same as input, or changed to another
particular type). I know that I can query the DataArray for its
contained type, but I suppose that doesn't help the compiler since
I've only restricted it (at the level that the compiler can see) to
struct/IConvertible (limitation of the compiler not able to apply a
Numeric constraint, which I think is silly).

That sounds like they should be generic methods then.

True. However I can't access that until I can figure out how to pass
around the variables and types needed.

Well again you *can* use reflection on generic methods, but it's a bit
painful. The point is that if you don't know at compile-time what the
type parameters are when you call Process, then generics aren't really
going to help you. Of course, if you're calling it from another generic
method you can use the type parameters again, but sooner or later
you're going to need compile-time information, or you'll be forced to
use reflection.

Well, I've tried a number of workarounds and various reflection
methodologies, and they all suck. Any that actually work introduce so
much overhead both in code complexity and speed that it's a waste to
even consider them over the current hodge-podge of internal array and
type juggling.

Generics are great where they actually work (and I use them in several
places where they really do help clean up the code), but it appears
that what I want to do here is just outside what they can do, fitting
into that area where templates worked great but generics can't cope.
A shame since it seemed at first glance to be such a fine match.

David

 

[Jon Skeet [C# MVP]]

Generics are great where they actually work (and I use them in several
places where they really do help clean up the code), but it appears
that what I want to do here is just outside what they can do, fitting
into that area where templates worked great but generics can't cope.
A shame since it seemed at first glance to be such a fine match.

Would templates definitely have worked here, even? The trouble is, you
don't know the input type. That means that anything which is meant to
be compile-time safe will have issues.

You could put in a reflective *layer* so that most of the code was
safe, but you just called it via something which did appropriate
reflection to create/use the appropriate generic type/method. Not
entirely satisfactory, but possibly better than nothing.