modifying memory usage of a program (F-Prot)?

[news]

It started with my noticing our fileserver closing all open connections
and restarting HTTPD and other processes each night.
According to the /var/log/messages, I'd been getting a slew of these
messages the same time each night:
Aug 15 05:03:21 fileserve kernel: Out of Memory: Killed process 30974
(httpd).
Aug 15 05:03:27 fileserve kernel: Out of Memory: Killed process 30975
(httpd).
Aug 15 05:03:32 fileserve kernel: Out of Memory: Killed process 7835
(sshd).
But there's nothing set to run at 5am each day.

With the help from the newsgroup, I set up a cron to dump "top" for a
file every 20 minutes, and discovered the F-prot anti-virus program
which starts it's check at 4am each morning, starting out taking up
very low memory and then more and more over time. Less and less free
swap and buffers, until finally an hour later, "Out of Memory" errors
and things start shutting down.

The fileserver has 512MB of RAM and 256MB of swap. (That's what Fedora
Core2's autoDisk Druid suggested to make it.) I know I can renice a
process to give it more processor priority...is there some way I can
tell F-prot to try to run using less memory? I really don't want to go
without a daily virus check, but this crashing of the server every
night is not a good thing either.

Thanks for any suggestions!
Liam

 

[Malke]

It started with my noticing our fileserver closing all open
connections and restarting HTTPD and other processes each night.
According to the /var/log/messages, I'd been getting a slew of these
messages the same time each night:
Aug 15 05:03:21 fileserve kernel: Out of Memory: Killed process 30974
(httpd).
Aug 15 05:03:27 fileserve kernel: Out of Memory: Killed process 30975
(httpd).
Aug 15 05:03:32 fileserve kernel: Out of Memory: Killed process 7835
(sshd).
But there's nothing set to run at 5am each day.

With the help from the newsgroup, I set up a cron to dump "top" for a
file every 20 minutes, and discovered the F-prot anti-virus program
which starts it's check at 4am each morning, starting out taking up
very low memory and then more and more over time. Less and less free
swap and buffers, until finally an hour later, "Out of Memory" errors
and things start shutting down.

The fileserver has 512MB of RAM and 256MB of swap. (That's what Fedora
Core2's autoDisk Druid suggested to make it.) I know I can renice a
process to give it more processor priority...is there some way I can
tell F-prot to try to run using less memory? I really don't want to go
without a daily virus check, but this crashing of the server every
night is not a good thing either.

Thanks for any suggestions!
Liam

Have you tried contacting F-Prot's tech support? If this is a known
issue, they would have information about workarounds, settings, etc.
I've run F-Prot on SuSE 9.2 acting as a server for some Windows boxen
and I use it on all my Windows machines with no problem. Do you have
the latest version of their software?

Malke

 

[John-Paul Stewart]

It started with my noticing our fileserver closing all open connections
and restarting HTTPD and other processes each night.
According to the /var/log/messages, I'd been getting a slew of these
messages the same time each night:
Aug 15 05:03:21 fileserve kernel: Out of Memory: Killed process 30974
(httpd).
Aug 15 05:03:27 fileserve kernel: Out of Memory: Killed process 30975
(httpd).
Aug 15 05:03:32 fileserve kernel: Out of Memory: Killed process 7835
(sshd).
But there's nothing set to run at 5am each day.

With the help from the newsgroup, I set up a cron to dump "top" for a
file every 20 minutes, and discovered the F-prot anti-virus program
which starts it's check at 4am each morning, starting out taking up
very low memory and then more and more over time. Less and less free
swap and buffers, until finally an hour later, "Out of Memory" errors
and things start shutting down.

The fileserver has 512MB of RAM and 256MB of swap. (That's what Fedora
Core2's autoDisk Druid suggested to make it.) I know I can renice a
process to give it more processor priority...is there some way I can
tell F-prot to try to run using less memory? I really don't want to go
without a daily virus check, but this crashing of the server every
night is not a good thing either.

Are you running a recent version of F-Prot? ISTR there being a problem
with a version leaking memory a couple of years ago. (I switched to
ClamAV though, so I'm not sure if it got fixed. I'm also not entirely
sure it was F-Prot that had the problem in the first place.) With the
buggy software, it didn't matter how much RAM you had; eventually the
memory leak would cause all of it to be used.

Also, check with the makers of F-Prot for support. You should be
entitled to support with your paid license.

As a quick fix, you could create a swap *file* to add extra swap space.
Find a filesystem with more than 1GB free space and 'cd' to a
convenient directory on it and execute (as root):

dd if=/dev/zero of=swapfile bs=1M count=1K
mkswap swapfile
swapon swapfile

I suspect that you'll still see the problem occur. It'll just take
longer to fill the extra swap space. (Use the 'swapoff' command to
disable the swap file, and then delete it when you're done with the
test.) However, if it doesn't happen with the extra swap file, then
you'll know that your machine was really just memory starved rather than
there being a memory leak in F-Prot. Make the extra swap space
permanent and/or add more RAM to the machine.

 

[news]

(e-mail address removed) wrote: ....is there some way I can

Are you running a recent version of F-Prot? ISTR there being a problem
with a version leaking memory a couple of years ago. (I switched to

Program version: 4.5.4
Engine version: 3.16.6

I think the latest is 4.6. I'm downloading it now.
I'm using the same version on two machines, the fileserver and my home
PC. The home PC hasn't had any problems, is also Fedora Core, and has
512MB RAM, but 1.5GB of swap! (Auto Disk Druid as well. Weird.)
So, I'm going to try....

As a quick fix, you could create a swap *file* to add extra swap space.
Find a filesystem with more than 1GB free space and 'cd' to a
convenient directory on it and execute (as root):

dd if=/dev/zero of=swapfile bs=1M count=1K
mkswap swapfile
swapon swapfile

Thanks for the suggestion!
BTW, I noticed once I did the "dd" line, my command prompt didn't
return. Just carriage returned. So I did the other two lines, and then
CTRL-C to return to the command prompt. Seemed to work OK, as I now
have a "swapfile" file that's 600MB large. But I MAN dd and didn't see
anything about entering some sub-shell when using the dd command. Did
CTRL-C mess anything up, you think?

Also, while it looks like I may need to keep the swapfile file, if I
want to deactivate it with swapoff, will that be all I need to type?
And it'll go back to using the original "partition"? I see "swaoff -a"
would be bad!
"swapoff swapfile" perhaps? I see the original swap in my /etc/fstab,
so if I just did "swapoff", that should work OK?

(Sorry for the inane questions, but I've never worked with the swapfile
before, and don't want to make some stupid mistake.)

Thanks!
Liam

 

[John-Paul Stewart]

Thanks for the suggestion!
BTW, I noticed once I did the "dd" line, my command prompt didn't
return. Just carriage returned.

You didn't wait long enough. Writing a gigabyte of zeroes takes a while.

So I did the other two lines, and then
CTRL-C to return to the command prompt. Seemed to work OK, as I now
have a "swapfile" file that's 600MB large. But I MAN dd and didn't see
anything about entering some sub-shell when using the dd command. Did
CTRL-C mess anything up, you think?

It just ended prematurely with 600MB instead of the 1GB had the command
completed.

Also, while it looks like I may need to keep the swapfile file, if I
want to deactivate it with swapoff, will that be all I need to type?
And it'll go back to using the original "partition"? I see "swaoff -a"
would be bad!
Indeed!

"swapoff swapfile" perhaps?

I guess. 'man swapoff' should tell you...I haven't had to do that in a
while so I don't remember and I'm too lazy to look it up.

 

[news]

You didn't wait long enough. Writing a gigabyte of zeroes takes a while.

Ah, yeah, I see. Warning to any other newbies reading this...be
patient! =)
I upgraded the software yesterday, still crashed last night. The "top"
output shows it ran out of buffers and swap...which was indeed the same
size is before.
So I deleted and followed your instructions again...properly, and
voila! The "top" now shows swap being 1GB larger, and "swapon -s" shows
that new file as part of the swap.
What a cool trick that is. =)
So I anticipate everything working OK tonight.
Thanks for the help!!
Liam

 

[John-Paul Stewart]

Ah, yeah, I see. Warning to any other newbies reading this...be
patient! =)

I really should have mentioned that in my first post. I've done it so
often that I tend to forget others may not have created such a large
file with dd before and thus may not know that it will take a while.

I upgraded the software yesterday, still crashed last night. The "top"
output shows it ran out of buffers and swap...which was indeed the same
size is before.
So I deleted and followed your instructions again...properly, and
voila! The "top" now shows swap being 1GB larger, and "swapon -s" shows
that new file as part of the swap.
What a cool trick that is. =)
So I anticipate everything working OK tonight.
Thanks for the help!!

It'll be interesting to see if the problem re-occurs, but later due to
the larger available swap space (indicating a memory leak in the app) or
if it goes away (indicating your system was just starved for memory in
which case you may want to add more RAM as well as the swap space.)

 

[Unruh]

It'll be interesting to see if the problem re-occurs, but later due to
the larger available swap space (indicating a memory leak in the app) or
if it goes away (indicating your system was just starved for memory in
which case you may want to add more RAM as well as the swap space.)

Yee gads. Get 1GB ofmemory and add that. swap is a hopeless way of creating
memory. It is about a factor of 1000 slower than memory. If you program
starts swappping you might as well forget about it. If you run out of swap,
you should definitley forget about it. Your machine will spend all of its
time swapping and none doing anything useful.
GET MORE MEMORY. Do not increase swap.it is useless for accomplishing
anything.

 

[Chiefy]

15 Aug 2005 14:48 UTC, (e-mail address removed) typed:

It started with my noticing our fileserver closing all open connections
and restarting HTTPD and other processes each night.
According to the /var/log/messages, I'd been getting a slew of these
messages the same time each night:
Aug 15 05:03:21 fileserve kernel: Out of Memory: Killed process 30974
(httpd).
Aug 15 05:03:27 fileserve kernel: Out of Memory: Killed process 30975
(httpd).
Aug 15 05:03:32 fileserve kernel: Out of Memory: Killed process 7835
(sshd).
But there's nothing set to run at 5am each day.

Wonder if you have an OOM killer enabled. Turning it off won't help you
find the problem, but it should stop other processes being killed off by
your rogue 'possibly' program.

 

[news]

15 Aug 2005 14:48 UTC, (e-mail address removed) typed:

Wonder if you have an OOM killer enabled. Turning it off won't help you
find the problem, but it should stop other processes being killed off by
your rogue 'possibly' program.

I have no clue. It's Fedora Core 2, with as few of the built-in
packages as I could get away with. (I may have built it just using the
"Server" setup path.) So, I don't even know where to start looking to
see if it has an OOM killer.
/var/log/messages doesn't give any indication WHAT is closing the
services when it runs out of memory, they just are.

Liam

 

[Michael Heiming]

Yee gads. Get 1GB ofmemory and add that. swap is a hopeless way
of creating memory. It is about a factor of 1000 slower than
memory. If you program starts swappping you might as well
forget about it. If you run out of swap, you should definitley
forget about it. Your machine will spend all of its time
swapping and none doing anything useful. GET MORE MEMORY. Do
not increase swap.it is useless for accomplishing anything.

Yep, more memory does certainly help and it's cheap enough.
However, I'd enable overcommit memory at first and see what that
does, some apps tend to allocate memory in a strange way.

 

[John-Paul Stewart]

Yee gads. Get 1GB ofmemory and add that. swap is a hopeless way of creating
memory. It is about a factor of 1000 slower than memory. If you program
starts swappping you might as well forget about it. If you run out of swap,
you should definitley forget about it. Your machine will spend all of its
time swapping and none doing anything useful.
GET MORE MEMORY. Do not increase swap.it is useless for accomplishing
anything.

If you look up thread, I was suggesting the OP add the swap as a
diagnostic tool. If he increases swap by 1GB and the original problem
(F-Prot using all RAM and swap) persists, that's a likely indication of
a memory leak or other problem with the application. In that case, no
amount of additional RAM will help. So why spend the money on extra RAM
until you know for sure that it will help?

However, if the problem does go away with the additonal swap, then I
fully agree that extra RAM is a better long-term solution.

 

[news]

If you look up thread, I was suggesting the OP add the swap as a
diagnostic tool. If he increases swap by 1GB and the original problem
(F-Prot using all RAM and swap) persists, that's a likely indication of
a memory leak or other problem with the application. In that case, no
amount of additional RAM will help. So why spend the money on extra RAM
until you know for sure that it will help?

However, if the problem does go away with the additonal swap, then I
fully agree that extra RAM is a better long-term solution.

Welp, it worked. I dumped "top" at 15 minute intervals, and at times
the used swap space was past what I used to have, but never came close
to all of the new swap space.
So the problem is indeed not the program, but the amount of RAM.
Thanks for the help!! It's greatly appreciated!
(I am definitely writing this trick down in my "cool tips" journal.)
Liam