Async File IO Implementation

M

Muj Beg

Hi All,
For various reasons, I have been trying to get a closer look into Async
file IO implementation of the framework. Am I correct in assuming that the
Rotor (http://www.dotnet247.com/247reference/System/IO/FileStream/__rotor)
implementation is a fair (if not exact) representation of the workings of
this class? If so, I am failing to see how async IO works!

For example, in the rotor code, the variable "_canUseAsync" seems to be
set to false all the time. Further, if we trace the call-graph of
FileStream.BeginWrite(), it always ends up calling WriteFileNative(),
passing null for the NativeOverlapped paramater!

Obvisouly, I am missing something here. But I can't figure out what that
is!

Thanks,
Muj Beg
 
P

Peter Duniho

Muj said:
Hi All,
For various reasons, I have been trying to get a closer look into
Async file IO implementation of the framework. Am I correct in assuming
that the Rotor
(http://www.dotnet247.com/247reference/System/IO/FileStream/__rotor)
implementation is a fair (if not exact) representation of the workings
of this class? If so, I am failing to see how async IO works!

[...]
Obvisouly, I am missing something here. But I can't figure out what
that is!
Click to expand...

I would say that if you accept that Rotor is not an exact representation
of the .NET Framework, then obviously there will be situations in which
the Rotor source code doesn't tell you the full story.

I would suggest that's the case here.

I don't know the specifics of how the async file i/o is handled, but it
could be similar to the Socket class, which means it uses i/o completion
ports.

All that said, while learning more about the inner workings of the .NET
Framework may be academically useful, it's not like you can do anything
practical with the information. If there's something specific about
implementing some kind of async i/o API you'd like to know, it might be
better to ask about that specifically rather than worry about whether
Rotor is a good enough representation of the .NET Framework implementation.

Pete
 
J

Jeffrey Tan[MSFT]

Hi Muj,

Yes, Rotor is not a full presentation view of .Net CLR. It is an academic
implement for CLI. As far as I know Rotor Asynchronize file I/O
implementation is not the same as CLR.

I am not sure what type of details do you want to know about the .Net CLR
Asynchronize file I/O. Based on my knowledge, Windows CLR will leverage the
OS I/O completion port multithreading mechanism to achieve the high
performance and the scalability for multiple CPU processors. This should be
a high performance design because I/O completion provided by OS kernel is
the best choice to implement high performance multithreading application,
IIS and other Microsoft server-side applications also leverage the I/O
completion port. "Feroze Daud[MSFT]" provided some insight in his reply to
the blog entry below:
http://blogs.msdn.com/kavitak/archive/2003/12/15/Async-I_2F00_O-and-I_2F00_O
-completion-ports.aspx

If you are curious about the advantage of I/O completion port mechanism,
"Jeffrey Richter" provided a best explanation in "Chapter 2" of his book
<Programming Server-Side Applications for Microsoft Windows 2000>.

If you wanted to employ the I/O completion port feature in your own
multithreading application, you may leverage Overlapped.Pack(),
Overlapped.Unpack() and ThreadPool.BindHandle() methods to achieve the
task. Note: this can only be used on Microsoft Windows OS.

Hope this helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.
 
M

Muj Beg

Thanks Jeffrey.

The reason I was looking into this was that, I was using async file
operations in some IO bound code where I was expecting better performance
than the synchronous version, but was not getting that. Part of this also
was driven by the fact that I needed to write a custom file stream class
that behaves in a particular (somewhat unusual) way. Since I have a good
amount of Win32 Async IO experience with C++, I was looking into the
implementation details.

Ayway, thanks for the feedback you provided. I am aware of the managed
Overlapped class, and as you suggested, will probably go in that direction.

Thanks
Muj Beg

"Jeffrey Tan[MSFT]" said:
Hi Muj,

Yes, Rotor is not a full presentation view of .Net CLR. It is an academic
implement for CLI. As far as I know Rotor Asynchronize file I/O
implementation is not the same as CLR.

I am not sure what type of details do you want to know about the .Net CLR
Asynchronize file I/O. Based on my knowledge, Windows CLR will leverage
the
OS I/O completion port multithreading mechanism to achieve the high
performance and the scalability for multiple CPU processors. This should
be
a high performance design because I/O completion provided by OS kernel is
the best choice to implement high performance multithreading application,
IIS and other Microsoft server-side applications also leverage the I/O
completion port. "Feroze Daud[MSFT]" provided some insight in his reply to
the blog entry below:
http://blogs.msdn.com/kavitak/archive/2003/12/15/Async-I_2F00_O-and-I_2F00_O
-completion-ports.aspx

If you are curious about the advantage of I/O completion port mechanism,
"Jeffrey Richter" provided a best explanation in "Chapter 2" of his book
<Programming Server-Side Applications for Microsoft Windows 2000>.

If you wanted to employ the I/O completion port feature in your own
multithreading application, you may leverage Overlapped.Pack(),
Overlapped.Unpack() and ThreadPool.BindHandle() methods to achieve the
task. Note: this can only be used on Microsoft Windows OS.

Hope this helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no
rights.
Click to expand...
 
M

Muj Beg

Sorry, meant to say "CPU bound" code, instead of "IO bound"...


Muj Beg said:
Thanks Jeffrey.

The reason I was looking into this was that, I was using async file
operations in some IO bound code where I was expecting better performance
than the synchronous version, but was not getting that. Part of this also
was driven by the fact that I needed to write a custom file stream class
that behaves in a particular (somewhat unusual) way. Since I have a good
amount of Win32 Async IO experience with C++, I was looking into the
implementation details.

Ayway, thanks for the feedback you provided. I am aware of the managed
Overlapped class, and as you suggested, will probably go in that
direction.

Thanks
Muj Beg

"Jeffrey Tan[MSFT]" said:
Hi Muj,

Yes, Rotor is not a full presentation view of .Net CLR. It is an academic
implement for CLI. As far as I know Rotor Asynchronize file I/O
implementation is not the same as CLR.

I am not sure what type of details do you want to know about the .Net CLR
Asynchronize file I/O. Based on my knowledge, Windows CLR will leverage
the
OS I/O completion port multithreading mechanism to achieve the high
performance and the scalability for multiple CPU processors. This should
be
a high performance design because I/O completion provided by OS kernel is
the best choice to implement high performance multithreading application,
IIS and other Microsoft server-side applications also leverage the I/O
completion port. "Feroze Daud[MSFT]" provided some insight in his reply
to
the blog entry below:
http://blogs.msdn.com/kavitak/archive/2003/12/15/Async-I_2F00_O-and-I_2F00_O
-completion-ports.aspx

If you are curious about the advantage of I/O completion port mechanism,
"Jeffrey Richter" provided a best explanation in "Chapter 2" of his book
<Programming Server-Side Applications for Microsoft Windows 2000>.

If you wanted to employ the I/O completion port feature in your own
multithreading application, you may leverage Overlapped.Pack(),
Overlapped.Unpack() and ThreadPool.BindHandle() methods to achieve the
task. Note: this can only be used on Microsoft Windows OS.

Hope this helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent
issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each
follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no
rights.
Click to expand...
Click to expand...
 
J

Jeffrey Tan[MSFT]

Hi Muj,

Thank you for the feedback the background info.

Yes, asynchronous I/O with multithreading is not always fast than
synchronous I/O; we have to use it in proper way to get the maximize
performance.

It is good to know you are experencd with Win32 Async I/O. If you meet any
further problem during investigation, please feel free to tell me, I will
work with you. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.
 
Top