System.IO.File.Copy + NTFS Streams + special ACLs

[Peter Gibbons]

Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".

File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?


Regards,

Peter

 

[Mike]

Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".

How are you copying the file? That sounds like a OVERLAPPED I/O
(IOCP) completion status. You are using some async copy method.

File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?

Good question.

Since streams are supports at the RTL level, I wonder if using a "."
at the end of the file name will do this? I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

One way for sure:

Create your own "COPYFILE" function opening the main file name,
creating a target and a loop to read/write X KBytes blocks. That for
sure will only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source
in a server side meta file (xxxx.wcx:wcc) but the main stream
(xxxx.wcx) is the compiled code. When the RTE opens, copies to cache
and runs it. It only sees the compiled stream (xxxx.wcx), not the source.

--

 

[Mike]

Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".

How are you copying the file? That sounds like a OVERLAPPED I/O
(IOCP) completion status. You are using some async copy method.

File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?

Good question.

Since streams are supports at the RTL level, I wonder if using a "."
at the end of the file name will do this? I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

One way for sure:

Create your own "COPYFILE" function opening the main file name,
creating a target and a loop to read/write X KBytes blocks. That for
sure will only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source
in a server side meta file (xxxx.wcx:wcc) but the main stream
(xxxx.wcx) is the compiled code. When the RTE opens, copies to cache
and runs it. It only sees the compiled stream (xxxx.wcx), not the source.

--

 

[Peter Gibbons]

Hello,

How are you copying the file? That sounds like a OVERLAPPED I/O (IOCP)
completion status. You are using some async copy method.

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".

Good question.

Since streams are supports at the RTL level, I wonder if using a "." at
the end of the file name will do this?

Adding a . doesn't make a difference in this case.

I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

I know it can be done with unmanages code / pinvoke but I'd like to
avoid it.

One way for sure:

Create your own "COPYFILE" function opening the main file name, creating
a target and a loop to read/write X KBytes blocks. That for sure will
only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source in
a server side meta file (xxxx.wcx:wcc) but the main stream (xxxx.wcx) is
the compiled code. When the RTE opens, copies to cache and runs it. It
only sees the compiled stream (xxxx.wcx), not the source.

--

Regards,

Peter

 

[Peter Gibbons]

Hello,

How are you copying the file? That sounds like a OVERLAPPED I/O (IOCP)
completion status. You are using some async copy method.

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".

Good question.

Since streams are supports at the RTL level, I wonder if using a "." at
the end of the file name will do this?

Adding a . doesn't make a difference in this case.

I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

I know it can be done with unmanages code / pinvoke but I'd like to
avoid it.

One way for sure:

Create your own "COPYFILE" function opening the main file name, creating
a target and a loop to read/write X KBytes blocks. That for sure will
only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source in
a server side meta file (xxxx.wcx:wcc) but the main stream (xxxx.wcx) is
the compiled code. When the RTE opens, copies to cache and runs it. It
only sees the compiled stream (xxxx.wcx), not the source.

--

Regards,

Peter

 

[Mike]

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".

+1. Does seem odd.

Adding a . doesn't make a difference in this case.

Right.

Rolling your own "CopyFile" function will work:

Sub CopyFile(byval src as string, tar as string)
Dim sr As New FileStream(src,FileMode.Open,FileAccess.Read)
Dim sw As New FileStream(tar,FileMode.Create,FileAccess.Write)
Dim b(4*1024) As Byte
Do
dim n as integer = sr.Read(b, 0, b.Length)
if n <= 0 then exit do
sw.write(b,0,n)
Loop
sw.Close()
sr.Close()
end sub

usage:

try
CopyFile("afs1.txt","afs2.txt")
catch ex as exception
WriteLine(ex.message)
WriteLine(ex.stacktrace)
end try

To create the test, I used notepad to create the AFS1.TXT file and the
meta stream

NotePad afs1.txt

I typed junk lines and saved.

NotePad afs1.txt:meta. <--- NOTE THE ENDING DOT!

I typed junk lines and saved.

You can confirm that a straight DOS copy will copy the streams:

copy afs1.txt junk.txt
notepad junk.txt:meta. <--- NOTE THE ENDING DOT!

You will see the meta data.

Then I ran the above applet to copy the file to afs2.txt, and tested
to see if the the meta stream was copied as well:

NotePad afs2.txt:meta. <--- NOTE THE ENDING DOT!

Notepad asked if you want to create it.

Hope this provides you the solution you seek.

--

 

[Mike]

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".

+1. Does seem odd.

Adding a . doesn't make a difference in this case.

Right.

Rolling your own "CopyFile" function will work:

Sub CopyFile(byval src as string, tar as string)
Dim sr As New FileStream(src,FileMode.Open,FileAccess.Read)
Dim sw As New FileStream(tar,FileMode.Create,FileAccess.Write)
Dim b(4*1024) As Byte
Do
dim n as integer = sr.Read(b, 0, b.Length)
if n <= 0 then exit do
sw.write(b,0,n)
Loop
sw.Close()
sr.Close()
end sub

usage:

try
CopyFile("afs1.txt","afs2.txt")
catch ex as exception
WriteLine(ex.message)
WriteLine(ex.stacktrace)
end try

To create the test, I used notepad to create the AFS1.TXT file and the
meta stream

NotePad afs1.txt

I typed junk lines and saved.

NotePad afs1.txt:meta. <--- NOTE THE ENDING DOT!

I typed junk lines and saved.

You can confirm that a straight DOS copy will copy the streams:

copy afs1.txt junk.txt
notepad junk.txt:meta. <--- NOTE THE ENDING DOT!

You will see the meta data.

Then I ran the above applet to copy the file to afs2.txt, and tested
to see if the the meta stream was copied as well:

NotePad afs2.txt:meta. <--- NOTE THE ENDING DOT!

Notepad asked if you want to create it.

Hope this provides you the solution you seek.

--

 

[Peter Gibbons]

Hello Mike,

thanks for the example code. I changed it a little to my needs.
I'd like to preserve the LastWriteTime like the normal copy functions
but I get a "System.UnauthorizedAccessException" due to the NTFS
Permissions because the user only has "Create file" permission in the
destination directory.


Sub CopyFileWithoutStreams(ByVal sourceFileName As String, ByVal
destFileName As String, Optional ByVal buffersize As Integer = &HFFFFUI)
Dim sr As New System.IO.FileStream(sourceFileName,
IO.FileMode.Open, IO.FileAccess.Read, IO.FileShare.Read, buffersize,
IO.FileOptions.SequentialScan)
Dim sw As New System.IO.FileStream(destFileName,
IO.FileMode.CreateNew, IO.FileAccess.Write, IO.FileShare.None,
buffersize, IO.FileOptions.SequentialScan Or IO.FileOptions.WriteThrough)
Dim b(buffersize) As Byte
Dim n As Integer
Do
n = sr.Read(b, 0, buffersize)
If n <= 0 Then Exit Do
sw.Write(b, 0, n)
Loop
sw.Close()
sr.Close()
System.IO.File.SetLastWriteTimeUtc(destFileName,
System.IO.File.GetLastWriteTimeUtc(sourceFileName))
End Sub

Regards,

Peter

 

[Peter Gibbons]

Hello Mike,

thanks for the example code. I changed it a little to my needs.
I'd like to preserve the LastWriteTime like the normal copy functions
but I get a "System.UnauthorizedAccessException" due to the NTFS
Permissions because the user only has "Create file" permission in the
destination directory.


Sub CopyFileWithoutStreams(ByVal sourceFileName As String, ByVal
destFileName As String, Optional ByVal buffersize As Integer = &HFFFFUI)
Dim sr As New System.IO.FileStream(sourceFileName,
IO.FileMode.Open, IO.FileAccess.Read, IO.FileShare.Read, buffersize,
IO.FileOptions.SequentialScan)
Dim sw As New System.IO.FileStream(destFileName,
IO.FileMode.CreateNew, IO.FileAccess.Write, IO.FileShare.None,
buffersize, IO.FileOptions.SequentialScan Or IO.FileOptions.WriteThrough)
Dim b(buffersize) As Byte
Dim n As Integer
Do
n = sr.Read(b, 0, buffersize)
If n <= 0 Then Exit Do
sw.Write(b, 0, n)
Loop
sw.Close()
sr.Close()
System.IO.File.SetLastWriteTimeUtc(destFileName,
System.IO.File.GetLastWriteTimeUtc(sourceFileName))
End Sub

Regards,

Peter

 

[Mike]

Ok, the user doesn't have "Modify" access? or Write Attributes?

So how are you going to handle that? Impersonate a higher level account?

If this is a system application, not just a end-user application, it
can make sense to raise the process credentials so you can do your
system work.

 

[Mike]

Ok, the user doesn't have "Modify" access? or Write Attributes?

So how are you going to handle that? Impersonate a higher level account?

If this is a system application, not just a end-user application, it
can make sense to raise the process credentials so you can do your
system work.

 

[Mike]

Pete,

Check these out:

Troubleshooting Exceptions: System.UnauthorizedAccessException
http://msdn.microsoft.com/en-us/library/18b8kx07.aspx

Code Access Security Basics
http://msdn.microsoft.com/en-us/library/33tceax8.aspx

See the Secure Class Libraries and Requesting permission links.

--

 

[Mike]

Pete,

Check these out:

Troubleshooting Exceptions: System.UnauthorizedAccessException
http://msdn.microsoft.com/en-us/library/18b8kx07.aspx

Code Access Security Basics
http://msdn.microsoft.com/en-us/library/33tceax8.aspx

See the Secure Class Libraries and Requesting permission links.

--

 

[Peter Gibbons]

Hello Mike,

the applikation runs as a service under a least privilege account that
only has the "create file" permission in the archive directory, not
more, not less. Elevation isn't possible and shouldn't be neccessary.
Files that don't have NTFS alternate data streams can be copied without
problems. The LastWriteTime is also kept.

Regards,

Peter

 

[Peter Gibbons]

Hello Mike,

the applikation runs as a service under a least privilege account that
only has the "create file" permission in the archive directory, not
more, not less. Elevation isn't possible and shouldn't be neccessary.
Files that don't have NTFS alternate data streams can be copied without
problems. The LastWriteTime is also kept.

Regards,

Peter