User Function - limit on arguments

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I have created a user function that averages values in columns for each
record. It seems to work fine up to a point. The table has 30 columns for
scores. I can pass 29 columns/values to the function but when I pass the
30th column I get a message that says teh expression is too complex.

I can't find anything on limitations of this type. The user function has a
ParamArray as its single argument.

Any thoughts?
 
Have you considered normalizing your table structure? Normalization is a
great solution for getting rid of complexity.
 
PFW said:
I have created a user function that averages values in columns for
each record. It seems to work fine up to a point. The table has 30
columns for scores. I can pass 29 columns/values to the function but
when I pass the 30th column I get a message that says teh expression
is too complex.

I can't find anything on limitations of this type. The user function
has a ParamArray as its single argument.

Any thoughts?

Any time you find yourself needing to aggregate across columns it most likely
means that your table design is incorrect. If you had a one table with a
one-to-many relationship to a second table and the second table had 30 ROWS of
records for scores then you could use a simple totals query grouping by the
Foreign Key and using Avg() to get the average.

That aside I don't know of any inherant limit on the number of values that can
be passed with a paramArray. It wouldn't surprise me that there is one, but 29
seems awfully low.
 
Rick said:
Any time you find yourself needing to aggregate across columns it most likely
means that your table design is incorrect. If you had a one table with a
one-to-many relationship to a second table and the second table had 30 ROWS of
records for scores then you could use a simple totals query grouping by the
Foreign Key and using Avg() to get the average.

Changing the design for the convenience of queries may result in a
significantly more complex design.

Simplifying the example, consider a table that holds Things. The
business rules are:

· each Thing must have exactly one 'minimum height value';
· each Thing must have exactly one 'maximum height value';
· for a given Thing, 'minimum height value' must be less than
'maximum height value'.

CREATE TABLE Things (
things_number INTEGER NOT NULL UNIQUE,
min_height_value INTEGER NOT NULL,
CHECK (min_height_value > 0),
max_height_value INTEGER NOT NULL,
CHECK (max_height_value > 0),
CHECK (min_height_value < max_height_value)
);

The 'must have exactly one' business rules are simply implemented using
NOT NULL. The above CHECK constraints are analogous to field- and
record-level validation rules. In summary, the business rules are easy
to implement as constraints using the simple tools available in the
Access user interface.

The average height calculation is also fairly simple e.g.

SELECT thing_number,
(min_height_value + max_height_value) * 0.5
AS average_height_value
FROM Things;

Scaling from two columns to 30 means extra typing but no real
complexity.

Now consider the proposed design where each 'height value' is a row
rather than a column:

CREATE TABLE Things (
things_number INTEGER NOT NULL,
height_type CHAR(3),
CHECK (height_type IN ('Min', 'Max')),
UNIQUE (things_number, height_type),
height_value INTEGER NOT NULL,
CHECK (height_value > 0)
);

The required query is now even simpler:

SELECT things_number, AVG(height_value)
FROM Things
GROUP BY things_number;

However, the above table constraints do not satisfy any of the business
rules e.g. a Thing could have a 'minimum height value' less than its
corresponding 'maximum height value' or the corresponding 'maximum
height value' could be absent from the table.

The only way I can see that the business rules can be implemented with
the revised design is via table level CHECK constraints e.g.

ALTER TABLE Things ADD
CONSTRAINT things__one_max_and_one_min
CHECK (NOT EXISTS (
SELECT T2.things_number, COUNT(*)
FROM Things AS T2
GROUP BY T2.things_number
HAVING COUNT(*) <> 2)
);

ALTER TABLE Things ADD
CONSTRAINT things__min_less_than_max
CHECK (NOT EXISTS (
SELECT *
FROM Things AS T1, Things AS T2
WHERE T1.things_number = T2.things_number
AND T1.height_type = 'Min'
AND T2.height_type = 'Max'
AND T1.height_value >= T2.height_value)
);

In addition to the complexity, there are several issues with
table-level CHECK constraints. For example, attempting to add data to
the table:

INSERT INTO Things (things_number, height_type, height_value)
VALUES (1, 'Min', 1);

results in the CHECK biting before being given a chance to INSERT the
second row (one could argue this is indicative of a single atomic fact
having been split across two rows, a serious design flaw). Without the
ability to defer the constraint, it must be dropped, the first row
inserted and the CHECK recreated before the second row is inserted, all
within a transaction (because you don't want the table to be left in a
state where the constraint has been dropped). Try doing that with a
bound form <g>.

The above example has two related values and has resulted in
significant added complexity when the 'attributes as columns' design is
changed is to 'attributes as rows'. The OP has 30 columns...

Let's face facts: you weren't proposing the OP write table-level CHECK
constraints, were you <g>? Validation rules are easier to write at row
(record) level rather than at table level, as demonstrated above. So,
without details about the business rules, how do you know the OP's
design can be changed in the way you propose without necessitating a
more complex design, perhaps one requiring table-level CHECK
constraints?

Jamie

--
 
My guess is that you might be exceeding the length of text allowed in a
"cell" in the query grid. The function should be able to handle that many
items.

I have one I just tested with 40 items passed in the parameter array and it
had no problems.

From MS Access help Specifications
Number of characters in a cell in the query design grid 1,024
 
Not that - total characters in the cell is 402.

John Spencer said:
My guess is that you might be exceeding the length of text allowed in a
"cell" in the query grid. The function should be able to handle that many
items.

I have one I just tested with 40 items passed in the parameter array and it
had no problems.

From MS Access help Specifications
Number of characters in a cell in the query design grid 1,024
 
PFW said:
I have created a [VBA] function
that averages values

when I pass the
30th [value] I get a message
that says teh expression is too complex.

The [VBA] function has a
ParamArray as its single argument.

Any thoughts?

Consider using a Jet SQL procedure (a.k.a. Access parameter query)
instead. The number of arguments is effectively unlimited and, unlike a
VBA argument list, there is negligible overhead to having, say, a
thousand parameters with the NULL value as default. For details, see

http://groups.google.com/group/microsoft.public.access/msg/94f21df876f357c3

Jamie.

--
 
Can I suggest that you post the procedure?

Also, is it always the 30th column that causes the problem? Or does it work
for columns 1-19, but not for columns 1-19 and 30?

Here is one that I wrote about 6 years ago to handle this problem with
non-normalized data I was getting from an external source. It will handle
at least 40 arguments in the parameter array. Since I didn't type the array
it will handle strings, numbers, and even dates (although the latter doesn't
make a lot of sense).

Public Function fGetMeanAverage(ParamArray Values())
'John Spencer UMBC CHPDM
'Last Update: April 5, 2000
'Calculates the arithmetic average (mean) of a group of values passed to it.
'Sample call: myAvg = GetMeanAverage("1","TEST","2", "3",4,5,6,0) returns 3
(21/7)
'Ignores values that cannot be treated as numbers.

Dim i As Integer, intElementCount As Integer, dblSum As Double
intElementCount = 0
dblSum = 0

For i = LBound(Values) To UBound(Values)
If IsNumeric(Values(i)) Then 'Ignore Non-numeric values
dblSum = dblSum + Values(i)
intElementCount = intElementCount + 1
End If
Next i

If intElementCount > 0 Then
fGetMeanAverage = dblSum / intElementCount 'At least one number in the
group of values
Else
fGetMeanAverage = Null 'No number in the group of values
End If

End Function
 
Jamie said:
Changing the design for the convenience of queries may result in a
significantly more complex design.

Simplifying the example, consider a table that holds Things. The
business rules are:

· each Thing must have exactly one 'minimum height value';
· each Thing must have exactly one 'maximum height value';
· for a given Thing, 'minimum height value' must be less than
'maximum height value'.

I don't buy your example. An attribute of 'minimum height value' or
'maximum height value' sounds like a specification, not a measurement.
Those I agree would be stored in fields, not rows, but I fail to come up
with a scenario where I would need to aggregate two or more different
specifications.

Now, if I were taking "Measurements" that I needed to compare to the
specifications then I would want those in many rows of a single field and I
can easily see where I would need to aggregate them.
 
Rick said:
I don't buy your example. An attribute of 'minimum height value' or
'maximum height value' sounds like a specification, not a measurement.
Those I agree would be stored in fields, not rows, but I fail to come up
with a scenario where I would need to aggregate two or more different
specifications.

If you have a issues buying into my fabricated example, let's try
something more arbitrary.

A Thing must have *exactly* 30 measurements. Modelling each as a column
in the same table makes the constraint easy to write (i.e. make every
column NOT NULL) and INSERT/UPDATE/DELETE operations easy to use e.g.

CREATE TABLE Things (
thingID INTEGER NOT NULL UNIQUE,
measurement_01 INTEGER NOT NULL,
measurement _02 INTEGER NOT NULL,
measurement _03 INTEGER NOT NULL,
....
meanurement_30 INTEGER NOT NULL
);

If you propose that each measurement should be a row, how would you
write effective constraints?

Jamie.

--
 
Jamie said:
If you have a issues buying into my fabricated example, let's try
something more arbitrary.

A Thing must have *exactly* 30 measurements. Modelling each as a
column in the same table makes the constraint easy to write (i.e.
make every column NOT NULL) and INSERT/UPDATE/DELETE operations easy
to use e.g.

CREATE TABLE Things (
thingID INTEGER NOT NULL UNIQUE,
measurement_01 INTEGER NOT NULL,
measurement _02 INTEGER NOT NULL,
measurement _03 INTEGER NOT NULL,
...
meanurement_30 INTEGER NOT NULL
);

If you propose that each measurement should be a row, how would you
write effective constraints?

Jamie.

Frankly I wouldn't worrry about writing "effective constraints" for that
requirement.

There are numerous examples of business rules that cannot be implemented
with constraints. For example "Every Sales Order must have at least one
line-item". Surely you wouldn't advocate that the line-item data be moved
into fields of the parent order so you could enforce that with a constraint.

Constraints are useful and have their place. Enforcing every rule with them
is not achievable.
 
Rick said:
Frankly I wouldn't worrry about writing "effective constraints" for that
requirement.

It was the only requirement I specified: worry about it or you're fired
<vbg>!

As discussed upthread, it could be implemented using a table-level
CHECK constraint.
Constraints are useful and have their place. Enforcing
every rule with them is not achievable.

I'm not saying *every* rule should be implemented in the database.

For example, in an 'employee' database there is in theory a rule that
no rows in the 'earnings history' should have a date that is before the
'employee date of birth' or the 'employment start date' but
implementing all such rules would cause the database to grind to a
halt.

However, I do think that the criteria for deciding whether or not to
implement a constraint should come from the business, rather from the
limitations of the SQL product. To continue the example, I think the
database should enforce the rule that periods in an employee's salary
history should not overlap, otherwise you would have no primary key.

If the choice of SQL product is immutable then I propose the table
design be changed, rather than simply omitting an important business
rule and merely hoping data integrity ensues.
There are numerous examples of business rules that cannot be implemented
with constraints. For example "Every Sales Order must have at least one
line-item".

Now it's my turn not to buy it <g>. Give me some more details (using
Northwind?) and I'm sure I can come up with the constraints.

Jamie.

--
 
Jamie said:
It was the only requirement I specified: worry about it or you're
fired <vbg>!

As discussed upthread, it could be implemented using a table-level
CHECK constraint.

Yes, but "could" <> "should".

To me an entity "Test" that has one or more "Measurements" associated with
it is best modelled with two tables. A requirement that each parent entity
must have a fixed number of related children does not justify moving that
data into the parent table. That would introduce the desirable ability to
use a constraint, but simultaneously introduces many undesirable aspects
that (IMO) would outweigh the advantage of using a constraint.
 
Rick said:
A requirement that each parent entity
must have a fixed number of related children does not justify moving that
data into the parent table. That would introduce the desirable ability to
use a constraint, but simultaneously introduces many undesirable aspects
that (IMO) would outweigh the advantage of using a constraint.

The 'fixed number of related rows' is an 'extreme' example but this is
what I've been getting at: because of certain limitations in the
product, there is a case in Access/Jet to 'denormalize' to be able to
write effective constraints in a OLTP application where data integrity
is of prime importance.

Jamie.

--
 
Here is my function - seems to be similar to yours...

Function MyAverage(ParamArray Vals()) As Variant
Dim i As Integer
Dim nScores As Integer
Dim nCount As Integer

nScores = 0
nCount = 0

For i = LBound(Vals) To UBound(Vals)
If Not IsNull(Vals(i)) Then
nScores = nScores + Vals(i)
nCount = nCount + 1
End If
Next i

If nCount > 0 Then
MyAverage = nScores / nCount
Else
MyAverage = Null
End If
End Function

The function is bein used as follows:

Table has columns as follows-
ID (autonumber)
Col1 .. Col30 (longinteger - no default value)

Some columns have values in them - some do not.

The query lists the ID and the Average - the intention is to pass each of
the Col1 ..Col30 to the function and it returns an average of only those
where a value is entered.

This has been tried on two different WinXP machines - one with XP SP1 &
Office 2003 SP1 - the other with XP SP2 & Office 2003 & latest SP (can't
recall number).

This example was meant to be a simplistic test for a real life situation
where an applicant is required to undergo a specified number of up to 30
standard tests. The function is only meant to return the average of tests
completed so far.

The real life example was failing (the table and query were a lot more
detailed) so I tried to be very simple - without luck.
 
Sorry - I forgot to say - it doesn't seem to matter which combination of
columns is included - it will handle 29 but not 30.
 
So, does the function work if you just call it in the immediate (debug)
window. That is

?
MyAverage(1,2,3,4,5,6,7,8,9,10,1,2,3,4,5,6,7,8,9,20,1,2,3,4,5,6,7,8,9,30,1,2,3,4,5,6,7,8,9,40)
- which works on my computer
?
MyAverage(10000,20000,30000,40000,50000,6,7,8,9,10,1,2,3,4,5,6,7,8,9,20,1,2,3,4,5,6,7,8,9,30,1,2,3,4,5,6,7,8,9,40)
- fails with an overflow error

I would suggest that you Dim nScores as LONG or Double instead of integer.
If the sum of the numbers you are sending the function exceed 32K (limit of
integer) then you will get an overflow error.
 
John,

There is no problem with it in the Immediate Window and I have tested it
with up to 40 arguments. The variable types are not at issue as the examples
that I am using involve small values in the range of 1 to 10.

The problem occurrs when the function is used in a query. Did you get a
chance to try that?
 
No, but then I don't have a table with that structure.

Built a table and tried your function with it and got an error that the
expression was too complex when I entered the 30th element into the
function. So it looks as if the query can't handle passing that many
arguments.

It looks as if you will have to normalize that data or write some vba that
will take the primary key of the row and build an internal query to get the
results. Something like the UNTESTED AIRCODE below - add error handling and
set objects to nothing as appropriate. (Also, other things you could do to
optimize this).

The best optimization would be a table redesign, but if you can't do that
....

Public Function myFunkyAverage(PKValue)
Dim dbAny As DAO.Database
Dim rstAny As DAO.Recordset
Dim strSQL As String
Dim iLoop As Integer
Dim dblSum As Double
Dim intElementCount As Integer

Set dbAny = DBEngine(0)(0)
strSQL = "Select field1, field2, field3, ... , Field40 FROM table2 where
PK = " & Chr(34) & PKValue & chr(34)
'Drop the Chr(34) if you primary key is a number field

Set rstAny = dbAny.OpenRecordset(strSQL)

For iLoop = 1 To rstAny.Fields.Count - 1
If IsNumeric(rstAny.Fields(iLoop)) Then 'Ignore Non-numeric
values
dblSum = dblSum + rstAny.Fields(iLoop)
intElementCount = intElementCount + 1
End If
Next iLoop

If intElementCount > 0 Then
myFunkyAverage = dblSum / intElementCount
Else
myFunkyAverage = Null 'No number in the group of values
End If

rstAny.close
Set dbAny = Nothing
End Function
 
Back
Top