SFM 5 secs delay

[Guest]

Sorry for cross-posting to msmq.setup which is inappropriate
for this question, but I didn't find the group among "managed"
newsgroups appropriate for it.

I found the following article on msdn. It describes my problem
in details and even offers resolution, except that I observe this
problem not on NT but on Win2000 server with the latest service
pack, so resolution can't be applied.

http://support.microsoft.com/default.aspx?scid=kb;en-us;235543

In short this is what happens:
Our PC app creates file in some folder accessible via SFM from Mac.
This file is not visible to our Mac app that uses Mac OS file API
until a delay of 1-5 seconds. The odd thing is that attempt to
create file with the same name gives "file already exists" yet,
attempt to open file or get its info gives "file not found"
during this delay. Even enumerating folder doesn't reveal this
file during delay. FlushVol doesn't help. Creating another temp
file in the directory (in the hope that this will refresh something)
doesn't help either. The problem is definitely on the server side
(first - everything is ok with Mac afp server, second - analysis
of network traffic shows this clearly).

Any hints how to address this problem and remove delay are appreciated.

Mike

 

[William Smith]

In short this is what happens:
Our PC app creates file in some folder accessible via SFM from Mac.
This file is not visible to our Mac app that uses Mac OS file API
until a delay of 1-5 seconds. The odd thing is that attempt to
create file with the same name gives "file already exists" yet,
attempt to open file or get its info gives "file not found"
during this delay. Even enumerating folder doesn't reveal this
file during delay. FlushVol doesn't help. Creating another temp
file in the directory (in the hope that this will refresh something)
doesn't help either. The problem is definitely on the server side
(first - everything is ok with Mac afp server, second - analysis
of network traffic shows this clearly).

Hi Mike!

It's not unusual for any networked machine to have a delay when
refreshing windows on a server. I'd say that 1-5 seconds is actually
pretty good. The client not the server will prompt the refresh.

With this in mind, have a look at a post I responded to a while back. It
points to someone's suggestion for using a very simple Applescript to
refresh the window on demand.
<http://groups-beta.google.com/group/microsoft.public.mac.otherproducts/m
sg/f486edc8e877a4d9?hl=en>

Hope this helps! bill

 

[Guest]

It's not unusual for any networked machine to have a delay when
refreshing windows on a server. I'd say that 1-5 seconds is actually
pretty good. The client not the server will prompt the refresh.

Well... First, no windows or other high level stuff is involved here,
I am talking about straight low-level APIs like fopen/CreateFile,
stat/GetFileAttributes, opendir/FindFirstFile, etc. Then, my Mac
and Win2K server are on the local gigabit LAN. From where I sit
delay of several seconds like this (actually any delay, because as
it turns out it has nothing to do with network traffic per se, see
below) is inappropriate in this case: analysis of network traffic
clearly shows that whenever the client API like "stat" is called
during this delay, the packet containing afpGetFlDrParms command
is sent to server and the server gives an immediate "file not found"
response. This seems very wrong, looks like SFM service (thing that
is *local* to that Win2K box) has a very different idea about file
system state than the actual file system state is. These states are
out of sync. As if SFM caches file information and updates it infrequently,
like once in 5 seconds. But at the same time, attempting to create file
with the same name properly fails (SFM somehow checks this particular
operation against real file system state, thing that it doesn't perform
with other commands for some reason). I still hope there is workaround
to this (short of not using SFM/AFP), it'd be very sad if there is no one.

Do you know appropriate managed newsgroup for this question, ideally
which is monitored by guys who do SFM?

Mike Kluev

 

[Jim Seifert [MSFT]]

The delay is due to the design of SFM as is the immediate response when you
try and create a duplicate file. When trying to create a duplicate file
from a Mac client the SFM service asks NTFS to try and create the file, then
the SFM service returns the NTFS error to the client. NTFS is aware of
files not yet included in the afpIndex. When a file is created on a SFM
shared volume another file is created for the resource fork, index
information is added to afpIndex and the index rebuilt. Both the size of
the index and Antivirus software monitoring NTFS streams can influence the
time this takes.

 

[Guest]

The delay is due to the design of SFM as is the immediate response when you
try and create a duplicate file. When trying to create a duplicate file
from a Mac client the SFM service asks NTFS to try and create the file, then
the SFM service returns the NTFS error to the client. NTFS is aware of
files not yet included in the afpIndex. When a file is created on a SFM
shared volume another file is created for the resource fork, index
information is added to afpIndex and the index rebuilt. Both the size of
the index and Antivirus software monitoring NTFS streams can influence the
time this takes.

Thanks Jim. I was awaiting and at the same time afraid of answer like
this. Given that this is by design, I have no choice but one of:

- Ask our customers to tolerate the delay (which is much more than
5 seconds because N files involved, and delays sum up).

- Ask our customers to not use SFM for file sharing with our app.

- Rewrite our app(s) so it does files in "parallel" resulting a single
5 sec delay instead of many.

Neither of the above is easy, so forgive me pursuing this a little further:

- Is the mentioned article related to this issue (aside it is NT specific):
http://support.microsoft.com/default.aspx?scid=kb;en-us;235543
In other words, could it be the case that the underlying file creation
API does "rename" internally, and this is this "rename" per se the
whole problem is due to?

- Can I do something on server to force index recreation? E.g. create
file differently? "touch" file or parent? "ping" SFM process? Delete
afpIndex file? (is it a file at all?) Other mumbo-jumbo?

- You've mentioned index size. What controls it? Is there size limit for
afpIndex? If there is, the ugly workaround would be filling index up with
contents of some temporary folder, thus wiping the obsolete info about
the folder in questing. Then when info about the folder is needed it will
be read directly from NTFS.

- What mechanism is used by SFM on Win2K to trigger index updating?
Change notifications? Polling? If polling, what's polling interval and is it
customizable?

- What's the real purpose of afpIndex? Is it just a historic implementation
artifact or does it really speed things up?

- Don't you think this is design flaw? For me, that "create" operation goes
directly to NTFS bypassing index while other operations like "delete" or
"get info" works via index is ... strange to say the least. It gives
inconsistent
view on file system state. Is the problem somehow addressed in Win 2003 sever?

Thanks,
Mike Kluev

 

[Jim Seifert [MSFT]]

Questions from most recent post and answers inline:

- Is the mentioned article related to this issue (aside it is NT
specific):
http://support.microsoft.com/default.aspx?scid=kb;en-us;235543
In other words, could it be the case that the underlying file creation
API does "rename" internally, and this is this "rename" per se the
whole problem is due to?

I don't believe this KB or the file rename case relates to the issue you are
seeing.

- Can I do something on server to force index recreation? E.g. create
file differently? "touch" file or parent? "ping" SFM process? Delete
afpIndex file? (is it a file at all?) Other mumbo-jumbo?

Can't suggest a index file trick and don't believe it would help if I could.

- You've mentioned index size. What controls it? Is there size limit for
afpIndex? If there is, the ugly workaround would be filling index up with
contents of some temporary folder, thus wiping the obsolete info about
the folder in questing. Then when info about the folder is needed it will
be read directly from NTFS.

The index is a mapping of resource fork and data fork files on the shared
volume. To respond to sfm client quires the service must return information
about both pieces of the file. Information must be read from NTFS to
construct this index in any case as the SFM clients need this mapping.

- What mechanism is used by SFM on Win2K to trigger index updating?
Change notifications? Polling? If polling, what's polling interval and is
it
customizable?

The SFM service has registered for change notifications with NTFS.

- What's the real purpose of afpIndex? Is it just a historic
implementation
artifact or does it really speed things up?

See above.

- Don't you think this is design flaw? For me, that "create" operation
goes
directly to NTFS bypassing index while other operations like "delete" or
"get info" works via index is ... strange to say the least. It gives
inconsistent
view on file system state. Is the problem somehow addressed in Win 2003
sever?

This is unchanged in Win 2003 Server. It is a NTFS file create that
triggers a resource fork creation and index entry by the SFM service. This
file create must happen first. Delete, get info, rename all can be
initiated after a query of the afpindex.

 

[Guest]

The delay is due to the design of SFM as is the immediate response when you
try and create a duplicate file. When trying to create a duplicate file
from a Mac client the SFM service asks NTFS to try and create the file, then
the SFM service returns the NTFS error to the client. NTFS is aware of
files not yet included in the afpIndex. When a file is created on a SFM
shared volume another file is created for the resource fork, index
information is added to afpIndex and the index rebuilt. Both the size of
the index and Antivirus software monitoring NTFS streams can influence the
time this takes.

Jim, thanks for your help.

No antivirus software is running on the server. I tested now with the
newly created empty share - same delay :-(

Mike