DataTable.Select(filter) Parameter too long?

[Filipe Correia]

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe

 

[Guest]

I'm not seeing any recursion issues based on the length of elements in the
filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);

 

[Jeff Dillon]

You really have a 64K parameter??

 

[Filipe Correia]

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe

I'm not seeing any recursion issues based on the length of elements in the
filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe

 

[Filipe Correia]

You really have a 64K parameter??

yep
It's a generated filter string. I have an array of rows from dataset "A"
and I need to select the same rows (the ones with the same primary key)
from dataset "B". So, I generate the filter based on one set of rows and
use it to select() the same rows from the other dataset, which results,
in some extreme cases, in a 64k parameter (or in a 54k one now, please
check my other post).

Filipe

 

[Jeff Dillon]

You should do all this on the server. You should never pass a parameter like
that...

 

[Filipe Correia]

You should do all this on the server. You should never pass a parameter like
that...

In this particular situation I'm doing this in memory because I'm
comparing some data on the server with some data on a local dataset. To
do that I' pulling the data I want from the server to a second dataset
and comparing both datasets in memory.

I'm also using long filters in other situations, but that's a long story...

Anyhow, I can't find anything on the docs about the select method having
a limit, so I'm not yet sure if this is a bug or not.


Filipe

 

[Filipe Correia]

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe

I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe

 

[Jeff Dillon]

Yep, shouldn't do that. Not a bug, you're doing it wrong. Let the engine do
what it's designed to do

 

[Guest]

That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe

I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:


Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe

 

[Filipe Correia]

I tried enclosing parts of the filter expression in parentisis in a way
it won't afect it's semantics, and it worked.
Not the best of solutions, but it's an efective workaround non the less.

Cheers,
Filipe

That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

:

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark Ashton wrote:


I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:



Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe

 

[Jeff Dillon]

Hack?

I tried enclosing parts of the filter expression in parentisis in a way it
won't afect it's semantics, and it worked.
Not the best of solutions, but it's an efective workaround non the less.

Cheers,
Filipe

That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

:

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------




Filipe Correia wrote:

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark Ashton wrote:


I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:



Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe