Memory leak during backup

[Ronald67]

I'm using SyncBackSE to backup my harddisk data (NTFS) to a network attached
harddrive (FAT32). Among this data is a large 12 Gb file. What happens during
backup (I think only during backup of this large file, but I'm not sure) is
that the page pool / kernel memory gets exhausted, which after a few backups
leads to my PC completely freezing. I've run PoolMon.exe to find out which
process allocates memory without freeing it, but the only thing I know so far
is that the pooltag responsible for the memory leak is 'NXFS'. I did not
manage to find out which process is using this pooltag, so I'm stuck in
finding out which process is causing the memory leak. Is there anyone who
experienced the same problem or knows which process uses the 'NXFS' pooltag?
This would help me in my quest to find a solution for this problem.

 

[Jose]

I'm using SyncBackSE to backup my harddisk data (NTFS) to a network attached
harddrive (FAT32). Among this data is a large 12 Gb file. What happens during
backup (I think only during backup of this large file, but I'm not sure) is
that the page pool / kernel memory gets exhausted, which after a few backups
leads to my PC completely freezing. I've run PoolMon.exe to find out which
process allocates memory without freeing it, but the only thing I know sofar
is that the pooltag responsible for the memory leak is 'NXFS'. I did not
manage to find out which process is using this pooltag, so I'm stuck in
finding out which process is causing the memory leak. Is there anyone who
experienced the same problem or knows which process uses the 'NXFS' pooltag?
This would help me in my quest to find a solution for this problem.

Before you start looking at memory leaks, you need to take into
account the maximum size of a file on a FAT32 volume:

http://support.microsoft.com/kb/314463

You cannot create a file larger than (2^32)-1 bytes (this is one byte
less than 4 GB) on a FAT32 partition.

You could consider switching the backup volume to NTFS where the file
limit is only limited by the volume size.

 

[Ronald67]

Sorry Jose, but that is not an issue. The 12 Gb file is copied to the NAS
without a problem and completely the same as the original. I don't know how
Windows XP handles NAS-devices (Freecom Network Drive, attached by using
Explorer - Map Network Drive) and the way they are formatted, but if the
FAT32 would be a problem, then it shouldn't be possible to have a file larger
than 4 Gb on the NAS. True?

Any other suggestions?

 

[Ken Blake, MVP]

Sorry Jose, but that is not an issue. The 12 Gb file is copied to the NAS
without a problem and completely the same as the original. I don't know how
Windows XP handles NAS-devices (Freecom Network Drive, attached by using
Explorer - Map Network Drive) and the way they are formatted, but if the
FAT32 would be a problem, then it shouldn't be possible to have a file larger
than 4 Gb on the NAS. True?

Although that's true, what Jose says is correct. You can not have a
FAT32 file larger than 4GB. So something must be wrong with what you
are saying here.

 

[Pegasus [MVP]]

Although that's true, what Jose says is correct. You can not have a
FAT32 file larger than 4GB. So something must be wrong with what you
are saying here.

I seem to remember that some third-party partitioning/formatting tools allow
you to go beyond the 4GB limit on FAT32 partitions.

 

[Jose]

Sorry Jose, but that is not an issue. The 12 Gb file is copied to the NAS
without a problem and completely the same as the original. I don't know how
Windows XP handles NAS-devices (Freecom Network Drive, attached by using
Explorer - Map Network Drive) and the way they are formatted, but if the
FAT32 would be a problem, then it shouldn't be possible to have a file larger
than 4 Gb on the NAS. True?

Any other suggestions?

I am not sure with a NAS device but it seems like it would be well
documented.

Do you have files greater that 4GB already on your NAS device and how
did they get there?

If you can't verify by reading the documentation, you can perform a
simple test it by just seeing if you can successfully copy a single 4GB
+ file to your destination (err on the side of generosity).

 

[Pegasus [MVP]]

Although that's true, what Jose says is correct. You can not have a
FAT32 file larger than 4GB. So something must be wrong with what you
are saying here.

Perhaps the OP uses exFAT: http://support.microsoft.com/kb/955704/.

 

[Ken Blake, MVP]

I seem to remember that some third-party partitioning/formatting tools allow
you to go beyond the 4GB limit on FAT32 partitions.

If so, I don't know about it. I'd appreciate any documentation you
could cite.

 

[Tim Slattery]

If so, I don't know about it. I'd appreciate any documentation you
could cite.

There's a variant of FAT32 that allows that. It's called exFat. See
the Wikipedia article at http://en.wikipedia.org/wiki/ExFAT. You have
to apply an update of some sort to use it in XP.

 

[Ken Blake, MVP]

There's a variant of FAT32 that allows that. It's called exFat. See
the Wikipedia article at http://en.wikipedia.org/wiki/ExFAT. You have
to apply an update of some sort to use it in XP.

OK, thanks--but to me, that's exFAT, and *not* FAT32.

 

[Tim Slattery]

OK, thanks--but to me, that's exFAT, and *not* FAT32.

Agreed. It's another variant of the basic FAT structure, to go along
with FAT12, FAT16, and FAT32.

 

[Twayne]

If so, I don't know about it. I'd appreciate any documentation you
could cite.

http://en.wikipedia.org/wiki/File_Allocation_Table:
32 bit version: 8 TB (with 32KB clusters).

Max volume size 32 MB 2 GB
4 GB with 64k clusters (not widely supported) 2 TB
8 TB (with 32KB clusters)


BUT: Max file size 4 GB minus 1 byte (or volume size if smaller) for 12,
16 and 32 bit clusters.


I'll leave it to you to find a better site with a better explanation of
it.

ONLY 3rd party ware can extend the filesize beyond 4 Gig. Data is on x
line, therefore max =~ data line x 10 to the x power. XP has no such
native abilities.

Have you EVER done any of your own research Bruce? I gave you a place
to start; now go and find out for yourself. But I'll bet you don't use
it.

Twayne

 

[Twayne]

OK, thanks--but to me, that's exFAT, and *not* FAT32.

lol, I KNEW you'd say that!
Jeex, he SAID it was a "variant"; I could have predicted your weaseling
around. You'll find it IS FAT32 if you bothered to look at it.

 

[Ronald67]

It's always interesting to see that posting a question leads to a discussion,
which has nothing to do with the problem I put down. Discussing if it's
possible to write a file bigger than 12 Gb to my NAS was NOT a problem (as I
mentioned, it was copied to my NAS correctly), but the whole discussion was
just about this subject. Didn't help me at all. Too bad.

In the meanwhile I found which software is responsible for the memory leak.
The memory pool tag 'NXFS' is introduced by ThreatFire
(http://www.threatfire.com). Removing this software makes the pool tag 'NXFS'
disappear. Reinstalling it, makes it appear again. Probably this software
hooks into the file system to analyse suspect behaviour. When making a backup
of large files, ThreatFire probably analyses a lot of data, allocating memory
without freeing it, leading to an exhausted memory pool. I reported the
problem and hope it will be solved.

I hope this feedback will help others too, who experience the same problem.

 

[Pegasus [MVP]]

It's always interesting to see that posting a question leads to a
discussion,
which has nothing to do with the problem I put down. Discussing if it's
possible to write a file bigger than 12 Gb to my NAS was NOT a problem (as
I
mentioned, it was copied to my NAS correctly), but the whole discussion
was
just about this subject. Didn't help me at all. Too bad.

In the meanwhile I found which software is responsible for the memory
leak.
The memory pool tag 'NXFS' is introduced by ThreatFire
(http://www.threatfire.com). Removing this software makes the pool tag
'NXFS'
disappear. Reinstalling it, makes it appear again. Probably this software
hooks into the file system to analyse suspect behaviour. When making a
backup
of large files, ThreatFire probably analyses a lot of data, allocating
memory
without freeing it, leading to an exhausted memory pool. I reported the
problem and hope it will be solved.

I hope this feedback will help others too, who experience the same
problem.

"Jose" wrote:

Thanks for the feedback. Where exactly did you see the pool tax NXFS? Would
Process Explorer (www.sysinternals.com) let you link it back to ThreatFire?
Or maybe "tasklist.exe /v"?

 

[Ronald67]

Thanks for the feedback. Where exactly did you see the pool tax NXFS? Would
Process Explorer (www.sysinternals.com) let you link it back to ThreatFire?
Or maybe "tasklist.exe /v"?

No, I did it by trial-and-error. Uninstalling a program, rebooting, checking
with poolmon.exe if the tag NXFS had disappeared. If not, reinstalling the
software and trying the next one. Because someone hinted it could be file
system related (because of the FS in the tag name), I started with
anti-virus-programs (which are continuously checking file activity as far as
I understand). After I uninstalled ThreatFire, the NXFS tag had disappeared.
After reinstalling it was there again, so I'm pretty sure this is the
connected program. I scanned all files within the ThreatFire directory for
the text 'NXFS', but no file contained this text string. This would be the
easyest way to find the program associated with the tag.

 

[Pegasus [MVP]]

No, I did it by trial-and-error. Uninstalling a program, rebooting,
checking
with poolmon.exe if the tag NXFS had disappeared. If not, reinstalling the
software and trying the next one. Because someone hinted it could be file
system related (because of the FS in the tag name), I started with
anti-virus-programs (which are continuously checking file activity as far
as
I understand). After I uninstalled ThreatFire, the NXFS tag had
disappeared.
After reinstalling it was there again, so I'm pretty sure this is the
connected program. I scanned all files within the ThreatFire directory for
the text 'NXFS', but no file contained this text string. This would be the
easyest way to find the program associated with the tag.

Thanks for the feedback. Depending on what tool you used to find the NXFS
string, you may not be able to locate it. It is possible that it is stored
in Unicode format, in which case there is a $00 before every character. If
this is still an issue then I'm happy to send you a VB Script program that
would find the string no matter what. Send your EMail address to
pegasus_fnlATyahooDOTcom.

 

[Pegasus [MVP]]

No, I did it by trial-and-error. Uninstalling a program, rebooting,
checking
with poolmon.exe if the tag NXFS had disappeared. If not, reinstalling the
software and trying the next one. Because someone hinted it could be file
system related (because of the FS in the tag name), I started with
anti-virus-programs (which are continuously checking file activity as far
as
I understand). After I uninstalled ThreatFire, the NXFS tag had
disappeared.
After reinstalling it was there again, so I'm pretty sure this is the
connected program. I scanned all files within the ThreatFire directory for
the text 'NXFS', but no file contained this text string. This would be the
easyest way to find the program associated with the tag.

I was not previously familiar with poolmon.exe but after running it on my
machine I am quite sure that the tags do not relate to executables in any
way. Windows appears to allocate them more or less alphabetically. Looking
for NXFS in the ThreatFire folder would therefore be quite useless. Try
using the Task Manager to monitor the amount of memory consumed by your
various applications.