Interesting performance odities

[Fernando Cacciola]

Hi,

I was initially benchmarking inlinine capabilities of C# in VS 2005 while I
stumble on the following surprising results.

Please take a look at the following complete sample (you can just copy+paste
and compile it yourself)

using System;

public interface I { void Do () ; }

public class A : I { public void Do () {} }

public struct B<T> : I { public void Do () {} }

public class C : I { public virtual void Do () {} }

public class Test
{
const int c = int.MaxValue / 10 ;

static public void Try( I f )
{
int s = Environment.TickCount ;
for( int i = 0 ; i < c ; ++ i )
f.Do();
int e = Environment.TickCount ;
Console.WriteLine(e-s);
}

static public void GTry<F>( F f ) where F : I
{
int s = Environment.TickCount ;
for( int i = 0 ; i < c ; ++ i )
f.Do();
int e = Environment.TickCount ;
Console.WriteLine(e-s);
}
}

class Program
{

static void Main(string[] args)
{
Test.Try ( new A() );
Test.GTry( new A() );
Test.Try ( new B<int>() );
Test.GTry( new B<int>() );
Test.Try ( new C() );
}

}
<<<<<<<<<<<

In that program there are 3 types:

A a non-generic interface
A non-generic class implementing the interface.
And a generic class also implementing the interface.

All the interface declares is a trival no-op method.

There are two _exactly equivalent_ methods which differ ONLY in that one
takes the interface non-generically, and the other takes any type
implementing the interface generically.
The methods just call the trivial function on the parameter a number of
times and outputs the ellapsed time taken.

Do you expect any difference in the times taken to call Do() in each case?

I certianly wouldn't (Do is not a virtual method and it's trivial)

Now compile in release mode (making sure the code is optimized) and run it
_outside the IDE_.

I get the following surprising results:

1515
1516
2531
140
2109

Why is this surprising?

A few things:

(1) The method from the non-generic class A is not inlined at all in the
calls.

(2) The method in the generic-class B is as slow as if it were virtual
(compare it to class C) when called from the non-generic interface
(something which doesn't happen with A)

(3) The method is ONLY fully inlined when called on the generic-class B (not
on the non-generic class A) AND from the a generic parameter (thus fully
retaining its dynamic type) rather than an interface.

Can anyone help me make sense out of it? [or spot a mistake in the
benchmarks?]

TIA

 

[Nicholas Paldino [.NET/C# MVP]]

Fernando,

See inline:

(1) The method from the non-generic class A is not inlined at all in the
calls.

Why should it be? You are passing an interface implementation. The
call on the interface is never going to be inlined by the JIT.

(2) The method in the generic-class B is as slow as if it were virtual
(compare it to class C) when called from the non-generic interface
(something which doesn't happen with A)

No, it is slow because you are passing a structure to B and it is
getting boxed in the non-generic version of the method. In the generic
version, the parameter is not boxed.

(3) The method is ONLY fully inlined when called on the generic-class B
(not on the non-generic class A) AND from the a generic parameter (thus
fully retaining its dynamic type) rather than an interface.

This has nothing to do with it being generic. Rather, it is a matter of
in B, the public method is exposed as the implementation, and it is not
virtual, so therefore, that method can be inlined (that's not to say that it
^is^ inlined, but it is possible that it can be since it is not virtual).

Hope this helps.

--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

Hi,

I was initially benchmarking inlinine capabilities of C# in VS 2005 while
I stumble on the following surprising results.

Please take a look at the following complete sample (you can just
copy+paste and compile it yourself)

using System;

public interface I { void Do () ; }

public class A : I { public void Do () {} }

public struct B<T> : I { public void Do () {} }

public class C : I { public virtual void Do () {} }

public class Test
{
const int c = int.MaxValue / 10 ;

static public void Try( I f )
{
int s = Environment.TickCount ;
for( int i = 0 ; i < c ; ++ i )
f.Do();
int e = Environment.TickCount ;
Console.WriteLine(e-s);
}

static public void GTry<F>( F f ) where F : I
{
int s = Environment.TickCount ;
for( int i = 0 ; i < c ; ++ i )
f.Do();
int e = Environment.TickCount ;
Console.WriteLine(e-s);
}
}

class Program
{

static void Main(string[] args)
{
Test.Try ( new A() );
Test.GTry( new A() );
Test.Try ( new B<int>() );
Test.GTry( new B<int>() );
Test.Try ( new C() );
}

}
<<<<<<<<<<<

In that program there are 3 types:

A a non-generic interface
A non-generic class implementing the interface.
And a generic class also implementing the interface.

All the interface declares is a trival no-op method.

There are two _exactly equivalent_ methods which differ ONLY in that one
takes the interface non-generically, and the other takes any type
implementing the interface generically.
The methods just call the trivial function on the parameter a number of
times and outputs the ellapsed time taken.

Do you expect any difference in the times taken to call Do() in each case?

I certianly wouldn't (Do is not a virtual method and it's trivial)

Now compile in release mode (making sure the code is optimized) and run it
_outside the IDE_.

I get the following surprising results:

1515
1516
2531
140
2109

Why is this surprising?

A few things:


Can anyone help me make sense out of it? [or spot a mistake in the
benchmarks?]

TIA

 

[Fernando Cacciola]

Fernando,

See inline:


Why should it be? You are passing an interface implementation. The
call on the interface is never going to be inlined by the JIT.

I surely expected methods called from interfaces to be inlined, but, well,
now I regret to know they won't.

No, it is slow because you are passing a structure to B and it is
getting boxed in the non-generic version of the method.

Are you saying that when you pass a value-type to a method accepting an
interface, the value is boxed?
I surely never expected that.
Also, even if you were right, the difference is 1 second and there is at
most just 1 single boxing when the function object is passed to the Try
method. So no, your explanation doesn't account for the difference.
See below if you disagree.

In the generic version, the parameter is not boxed.


This has nothing to do with it being generic. Rather, it is a
matter of in B, the public method is exposed as the implementation,
and it is not virtual, so therefore, that method can be inlined
(that's not to say that it ^is^ inlined, but it is possible that it
can be since it is not virtual).

To me it has to do with it being generic (the compiler sees the
implementation _precisely_ because its not dealing with an interface but a
concrete type) (that's why I mentioned that its dynamic type was fully
retained)
Maybe I should have said that this very particular result was the only one I
expected.

I changed my code so that B is not a struct anymore but a class (that was a
mechanical mistake from years of C++ sample snippets)

These are the new results:

1484
1484
2078
3235
2062

Notice that

Test.Try ( new B<int>() );

Is still as slow as the virtual call.

But... What happened to the greately inlined call!!??
Why is


Test.GTry ( new B<int>() );

now taking _ven more_than the virtual call? I just changed B from a struct
to a class.

Puzzled

 

[Nicholas Paldino [.NET/C# MVP]]

Fernando,

I surely expected methods called from interfaces to be inlined, but, well,
now I regret to know they won't.

Nope, the JIT will not inline them.

Are you saying that when you pass a value-type to a method accepting an
interface, the value is boxed?
Absolutely.

Also, even if you were right, the difference is 1 second and there is at
most just 1 single boxing when the function object is passed to the Try
method. So no, your explanation doesn't account for the difference.
See below if you disagree.

There is only one boxing, but you have to remember, everything counts in
large amounts. Your math is a little off.

There is a difference of 1016 ticks between the two. What you are not
factoring in is that there are 214,748,364 calls occuring in the method!!!!
If you divide the 1016 ticks by that, you come out with 4.73e-6 which is
4.73e-9 extra seconds per call! That's a little less than one billionth of
a second.

Yes, you are right, it is not because of boxing, but you can't just
ignore the number of times you iterate and not factor it back into a
per-call number.

Also, you should use the new Stopwatch class in System.Diagnostics in
order to time these things. The counter is based on the performance
counter, which is more accurate.

To me it has to do with it being generic (the compiler sees the
implementation _precisely_ because its not dealing with an interface but a
concrete type) (that's why I mentioned that its dynamic type was fully
retained)
Maybe I should have said that this very particular result was the only one
I expected.

Yes, but it still has nothing to do with being generic. If the method
is not virtual, or an interface implementation, then there is no chance that
it will be inlined.

 

[James Curran]

I surely expected methods called from interfaces to be inlined, but, well,
now I regret to know they won't.

As the compiler would not know what type of object that method would be
called with, how could it *possibly* inline the call?

Are you saying that when you pass a value-type to a method accepting an
interface, the value is boxed?

Again, the compiler knows *nothing* about the object being passed, other
than it implements that interface. It has to assume that sometimes it will
be passed a reference-type (which, in your very example, sometime it is).
Hence, everything passed to it must be (or be made into) a reference-type.
Remember, there is one one copy of Test.Try(), and it must handle both As
and B<T>s.

--
Truth,
James Curran
[erstwhile VC++ MVP]

Home: www.noveltheory.com Work: www.njtheater.com
Blog: www.honestillusion.com Day Job: www.partsearch.com

 

[Fernando Cacciola]

Again, the compiler knows *nothing* about the object being passed,
other than it implements that interface. It has to assume that
sometimes it will be passed a reference-type (which, in your very
example, sometime it is). Hence, everything passed to it must be (or
be made into) a reference-type. Remember, there is one one copy of
Test.Try(), and it must handle both As and B<T>s.

Doesn't the JIT knows the exact target address of the called function
through the interface?
If not, what's the difference with a virtual function?
Or is it that interfaces are just abstract base classes in disguise?

TIA

 

[Christoph Nahr]

Yes, but it still has nothing to do with being generic. If the method
is not virtual, or an interface implementation, then there is no chance that
it will be inlined.

I think you meant to say here: "If the method _is_ virtual..."

 

[James Curran]

Doesn't the JIT knows the exact target address of the called function
through the interface?
If not, what's the difference with a virtual function?
Or is it that interfaces are just abstract base classes in disguise?

All that's irrelevant, because (as I repeat): There is one copy of
Test.Try(), and it must handle both As and B<T>s.

Sure, the JIT knows that exact target address, but it both cases, it's
the same address.

--
Truth,
James Curran
[erstwhile VC++ MVP]

Home: www.noveltheory.com Work: www.njtheater.com
Blog: www.honestillusion.com Day Job: www.partsearch.com

 

[Mike]

All that's irrelevant, because (as I repeat): There is one copy of
Test.Try(), and it must handle both As and B<T>s.

Sure, the JIT knows that exact target address, but it both cases, it's
the same address.

Thus pointing out the difference between generics (in this case boxing and
calling the same function) and templates (creating a specialized function
for each) -- this may be the OP's confusion.

--
Truth,
James Curran
[erstwhile VC++ MVP]

Home: www.noveltheory.com Work: www.njtheater.com
Blog: www.honestillusion.com Day Job: www.partsearch.com

 

[Jon Skeet [C# MVP]]

Thus pointing out the difference between generics (in this case boxing and
calling the same function) and templates (creating a specialized function
for each) -- this may be the OP's confusion.

No, the generic version doesn't have the boxing. It's the non-generic
version which has the boxing.

In generics, each reference type gets the same implementation (which
then can't do inlining of interface methods) but each value type gets a
different implementation (which is able to inline).

 

[Fernando Cacciola]

All that's irrelevant,

I'ts irrelevant as far as the discussion about inlining goes. But that was
just really a comment of mine (I tried to explain why the results were odd
to me).
The important thing of this whole thread which is going unnoticed is that
the generic's type method performs as if it where a virtual function _even_
if called in a generic method (that is, even if seen in its actual type).

So, forget about inlining.

Can you explain why all these 3 methods take the same average time?
Or at least, don't you find it surprising and odd?

Test.Try ( new B<int>() );
Test.GTry( new B<int>() );

Test.Try ( new C() );

TIA