A method of reducing on-demand scanning times

[null]

Here's an idea for users of DOS antivirus scanners on FAT drives. I
have a new DOS utility I call F-RECENT which finds only files written
to a drive in the past zero to nine days. It creates a text file
containing the fully qualified paths in short 8.3 form suitable for
use with both KAVDOS32 and McAfee's SCAN. Both of these scanners
accept such a text file, and they will scan only files listed.

My web site now shows a couple of sample batch files which can be
modified to add additional scanner switches as desired. The batches
merely serve as a demo of the method.

So now there can be no complaints about these scanners taking too long
to scan a drive Just scan more often .... and more thoroughly! It
all goes pretty quickly this way.

Art

http://home.epix.net/~artnpeg

 

[David W. Hodgins]

Here's an idea for users of DOS antivirus scanners on FAT drives. I
have a new DOS utility I call F-RECENT which finds only files written
to a drive in the past zero to nine days. It creates a text file
containing the fully qualified paths in short 8.3 form suitable for
use with both KAVDOS32 and McAfee's SCAN. Both of these scanners
accept such a text file, and they will scan only files listed.

Art, while that may be usefull in many/most cases, given that it's
a dos/fat system, any malware can alter the dates in the directory.

I'd be concerned that the target audience, that would use such
a utility (instead of scanning everything they download manually),
would get a false sense of security, from it's use.

Regards, Dave Hodgins

 

[Ian Kenefick]

Here's an idea for users of DOS antivirus scanners on FAT drives. I
have a new DOS utility I call F-RECENT which finds only files written
to a drive in the past zero to nine days. It creates a text file
containing the fully qualified paths in short 8.3 form suitable for
use with both KAVDOS32 and McAfee's SCAN. Both of these scanners
accept such a text file, and they will scan only files listed.

My web site now shows a couple of sample batch files which can be
modified to add additional scanner switches as desired. The batches
merely serve as a demo of the method.

So now there can be no complaints about these scanners taking too long
to scan a drive Just scan more often .... and more thoroughly! It
all goes pretty quickly this way.

Art

http://home.epix.net/~artnpeg

It's a great idea if you use FAT. It's like the stone age method of
checksumming and alternative data streams as used by some AV nowadays
to reduce inefficient re-scanning of unchanged files.

I was not being sarcastic by the way. I genuinely think those with an
older system would benefit greatly from reduced scan times and
overhead.

Great stuff - now If I can find my windows 98 disk I will give it a
go.

 

[James Egan]

Art, while that may be usefull in many/most cases, given that it's
a dos/fat system, any malware can alter the dates in the directory.

I'd be concerned that the target audience, that would use such
a utility (instead of scanning everything they download manually),
would get a false sense of security, from it's use.

Yes. It sounds like a repackaged version of last years utility which
looked like it stored hashes but actually stored simple modification
times. Far too easy to defeat and thus a false sense of security
creator.


Jim.

 

[null]

Art, while that may be usefull in many/most cases, given that it's
a dos/fat system, any malware can alter the dates in the directory.

I can throw the "useful in just many or most cases" argument back at
you since your recommended approaches to people suffering from
infestations never or rarely involves formal scanning

You're missing the whole idea, I think. Prevention is the name of the
game, not recovering from infestations. You don't allow malware to
act and get a chance to change anything! Or at least I don't

I'd be concerned that the target audience, that would use such
a utility (instead of scanning everything they download manually),
would get a false sense of security, from it's use.

Ah, but scanning everything they download manually is precisely where
this approach is superior. Most people don't even know what they've
inadvertently downloaded or what folder it may be in. Using my "recent
files" filtered method routinely every few days would show up new
downloads anywhere they might reside, and not just in some download
folder.

Think prevention and safe hex!

BTW, a batch file for the DOS scanners can contain a line in the
beginning invoking WGET to update the def files. Then put the batch
in a task scheduler

Art

http://home.epix.net/~artnpeg

 

[null]

Yes. It sounds like a repackaged version of last years utility which
looked like it stored hashes but actually stored simple modification
times.

Wrong. That was a entirely different thing. F-RECENT doesn't store
anything. It finds files recently written to disk.

Art

http://home.epix.net/~artnpeg

 

[James Egan]

Ah, but scanning everything they download manually is precisely where
this approach is superior. Most people don't even know what they've
inadvertently downloaded or what folder it may be in. Using my "recent
files" filtered method routinely every few days would show up new
downloads anywhere they might reside, and not just in some download
folder.

It would be useful if you could guarantee that any malware is scanned
before being executed, but you can't. Consequently, the user may well
be happily scanning bona-fide files while the malware changes the
modification time of itself so that it is never scanned and the user
is non the wiser.

Maybe you are planning on using it as one of a cocktail of measures
and not just in isolation as a replacement procedure but you need to
be clear about that otherwise the false sense of security scenario
comes to fruition.


Jim.

 

[null]

It would be useful if you could guarantee that any malware is scanned
before being executed, but you can't.

Huh? The whole idea of scanning downloads is to scan before execution.
This has long been a sensible "safe hex" recommendation. And it's far
more effective when more than one av is used on-demand. And it's far
more more effective to wait a couple of days because of the "day zero"
effect (avs not yet having updates available).

You're not making any sense at all.

Art

http://home.epix.net/~artnpeg

 

[Norman L. DeForest]

Wrong. That was a entirely different thing. F-RECENT doesn't store
anything. It finds files recently written to disk.

Just out of curiosity, what language is it written in?

I can do the same thing with one command-line in 4DOS, the (now) freeware
replacement for COMMAND.COM). To see all new files on the current drive:

dir /[d-2] /a /s /f /x \*.*

"dir" DIR command.
"/[d-2]" restrict to files date-stamped 2 days ago or later.
"/a" include all files and/or directories, even if hidden.
"/s" recurse through subdirectories.
"/f" just print full path and filename for each file,
"/x" using short filenames.
"\*.*" all filenames (assumed anyway if left off) starting from the
root directory.

Changing "*.*" to "c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*"
dir /[d-2] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*
gave me a list of all of the recently changed files on all my drives.

Defining "new" as an alias for that would allow you to type
new >newlist
and get a list of new files. Making the number of days a parameter
by using %1 instead of 2 in the alias definition (and using back-quotes
around the definition to stop "%1" from being expanded immediately):

alias new=`dir /[d-%1] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*`

new 2 >lessthan2daysold.txt

new 8 >lessthan8daysold.txt

Getting 4DOS has been tip number one on my Computer Tips page ever sinc
the page was created, back when 4DOS was shareware. Now that it's
freeware, there is no reason not to try it. See:
http://www.chebucto.ns.ca/~af380/Tips.html#Tip001

 

[null]

Just out of curiosity, what language is it written in?

QB4.5 and assembly langage.

I can do the same thing with one command-line in 4DOS, the (now) freeware
replacement for COMMAND.COM). To see all new files on the current drive:

dir /[d-2] /a /s /f /x \*.*

"dir" DIR command.
"/[d-2]" restrict to files date-stamped 2 days ago or later.
"/a" include all files and/or directories, even if hidden.
"/s" recurse through subdirectories.
"/f" just print full path and filename for each file,
"/x" using short filenames.
"\*.*" all filenames (assumed anyway if left off) starting from the
root directory.

Changing "*.*" to "c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*"
dir /[d-2] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*
gave me a list of all of the recently changed files on all my drives.

Defining "new" as an alias for that would allow you to type
new >newlist
and get a list of new files. Making the number of days a parameter
by using %1 instead of 2 in the alias definition (and using back-quotes
around the definition to stop "%1" from being expanded immediately):

alias new=`dir /[d-%1] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*`

new 2 >lessthan2daysold.txt

new 8 >lessthan8daysold.txt

Getting 4DOS has been tip number one on my Computer Tips page ever sinc
the page was created, back when 4DOS was shareware. Now that it's
freeware, there is no reason not to try it. See:
http://www.chebucto.ns.ca/~af380/Tips.html#Tip001

I'm not surprised that a batch with 4DOS can be designed to display
the files. Can you also get it to write a text file containing all the
fully qualified paths for the av use I suggested?

I found that LFN names were ok with McAfee's SCAN but that KAVDOS32
requires 8.3 names. The nice aspect of writing programs rather than
batches is that you usually have far more degrees of freedom. Quite
often, batches require "helper" programs to accomplish the goal.

You can also use Charles Dye's excellent LOCATE.COM to find recent
files, as well as to do quite a number of other useful things.
However, when you want the output to be in a specific kind of form, or
maybe the screen presentations to be a certain way, you have to write
your own code anyway.

In this case, I had long ago developed the core Folder/File find code,
so it was natural and convenient to use it to develop other programs.

Art


http://home.epix.net/~artnpeg

 

[James Egan]

Huh? The whole idea of scanning downloads is to scan before execution.
This has long been a sensible "safe hex" recommendation. And it's far
more effective when more than one av is used on-demand. And it's far
more more effective to wait a couple of days because of the "day zero"
effect (avs not yet having updates available).

lol. Why have you posted some irrelevant "safe hex" recommendation
about downloaded files?

Back on topic, you have created a utility which is supposedly a
"method of reducing on-demand scanning times". It rather defeats the
object of scanning in the first place if the malware isn't on the list
of files to be scanned.

Anyways, you didn't listen last time with your earlier version so I
can't really expect you to listen now.


Jim.

 

[Roger Wilco]

lol. Why have you posted some irrelevant "safe hex" recommendation
about downloaded files?

Back on topic, you have created a utility which is supposedly a
"method of reducing on-demand scanning times". It rather defeats the
object of scanning in the first place if the malware isn't on the list
of files to be scanned.

Why wouldn't it be on the list? Wouldn't a newly created file be on the
list? If something changes the date - you have other issues, no?

 

[null]

lol. Why have you posted some irrelevant "safe hex" recommendation
about downloaded files?

Back on topic, you have created a utility which is supposedly a
"method of reducing on-demand scanning times". It rather defeats the
object of scanning in the first place if the malware isn't on the list
of files to be scanned.

Anyways, you didn't listen last time with your earlier version so I
can't really expect you to listen now.

If a user is hit with malware that he/she didn't purposely Run, _and_
that malware happens to attempt modification date obfuscation as a
means of hiding from Recent Date heuristic detection methods, then
obviously Recent Date heuristics might fail to detect parts of (but
maybe not all of) the malware's files.

Is that what you're on about? If so, why not list some known malwares
that have successfully eluded Recent Date heuristics to make your
point? Lacking such evidence, and merely speculating on the
signficance, isn't very convincing to me

Art

http://home.epix.net/~artnpeg

 

[null]

BTW, on the subject of Recent File heuristics and malware obfuscation,
I recall that during the early development of my D-browse program that
Windows didn't allow changing a file's creation and last accessed
dates .... only the recently modified date. Yet, all three dates can
be changed in plain DOS. I didn't pursue this, just made a mental
note. It may be possible using low level interrupts to bypass Windows'
protection of the two dates, I dunno. But I do know it couldn't be
done using the LFN interrupt services. And I'll add that my work is
strictly with Win 9X/ME.

Now, maybe something useful can be done along these lines. I'm
thinking of the "last accessed" date. If that date is indeed
unalterable in Windows by any means, it provides the basis of a much
stronger Recent File Date heuristic.

Art

http://home.epix.net/~artnpeg

 

[kurt wismer]

Wrong. That was a entirely different thing. F-RECENT doesn't store
anything. It finds files recently written to disk.

not to put too fine a point on it but that's still got the same
problem... the data you're using to decide what's new or not (and
therefore what to scan) can be easily forged/modified/whatever...

 

[Norman L. DeForest]

Date: Tue, 12 Apr 2005 21:04:26 GMT
From: (e-mail address removed)
Newsgroups: alt.comp.anti-virus, alt.comp.virus
Subject: Re: A method of reducing on-demand scanning times

Just out of curiosity, what language is it written in?

QB4.5 and assembly langage.

I can do the same thing with one command-line in 4DOS, the (now) freeware
replacement for COMMAND.COM). To see all new files on the current drive:

dir /[d-2] /a /s /f /x \*.*

"dir" DIR command.
"/[d-2]" restrict to files date-stamped 2 days ago or later.
"/a" include all files and/or directories, even if hidden.
"/s" recurse through subdirectories.
"/f" just print full path and filename for each file,
"/x" using short filenames.
"\*.*" all filenames (assumed anyway if left off) starting from the
root directory.

Changing "*.*" to "c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*"
dir /[d-2] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*
gave me a list of all of the recently changed files on all my drives.

Defining "new" as an alias for that would allow you to type
new >newlist
and get a list of new files. Making the number of days a parameter
by using %1 instead of 2 in the alias definition (and using back-quotes
around the definition to stop "%1" from being expanded immediately):

alias new=`dir /[d-%1] /a /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*`

new 2 >lessthan2daysold.txt

new 8 >lessthan8daysold.txt

Getting 4DOS has been tip number one on my Computer Tips page ever sinc
the page was created, back when 4DOS was shareware. Now that it's
freeware, there is no reason not to try it. See:
http://www.chebucto.ns.ca/~af380/Tips.html#Tip001

I'm not surprised that a batch with 4DOS can be designed to display
the files. Can you also get it to write a text file containing all the
fully qualified paths for the av use I suggested?

Something like this (all items less than 2 days old)? (With snippage to
reduce post size and one comment about a correction.)

[snip]
C:\WINDOWS\CWBAUDIO.WCM
C:\WINDOWS\NDISLOG.TXT
C:\WINDOWS\OFFITEMS.LOG
C:\WINDOWS\SYSTEM.DAT
C:\WINDOWS\SYSTEM.INI
C:\WINDOWS\USER.DAT
C:\WINDOWS\WIN.INI
C:\WINDOWS\APPLIC~1\MICROS~1\HTMLHE~1\HH.DAT
C:\WINDOWS\APPLIC~1\MICROS~1\INTERN~1\DESKTOP.HTT
C:\WINDOWS\APPLIC~1\MOZILLA\FIREFOX\PROFILES\DEFAULT.TX4\BOOKMA~1.BAK
C:\WINDOWS\APPLIC~1\MOZILLA\FIREFOX\PROFILES\DEFAULT.TX4\BOOKMA~1.HTM
[snip]
C:\WINDOWS\APPLIC~1\TALKBACK\MOZILL~1\FIREFO~1\WIN32\200411~1\PERMDATA.BOX
C:\WINDOWS\APPLOG\APPLOG.IND
[snip]
C:\WINDOWS\COOKIES\INDEX.DAT
C:\WINDOWS\HISTORY\HISTORY.IE5\INDEX.DAT
C:\WINDOWS\LOCALS~1\APPLIC~1\MICROS~1\INTERN~1\MSIMGSIZ.DAT
C:\WINDOWS\LOCALS~1\TEMPOR~1\CONTENT.IE5\INDEX.DAT
C:\WINDOWS\PROFILES\ALLUSE~1\ADOBE\WEBBUY\HISTORY.TXT
C:\WINDOWS\RECENT\APPLE-~5.LNK
[snip]
C:\WINDOWS\RECENT\TEST4B~1.LNK
C:\WINDOWS\SENDTO\SHORTC~1.PIF
C:\WINDOWS\SYSBCKUP\RB002.CAB
C:\WINDOWS\SYSBCKUP\RB003.CAB
C:\WINDOWS\SYSTEM\FFASTLOG.TXT
D:\UNICODE
[include '-d' after the /a to suppress listing new directories]
D:\WIN386.SWP
D:\RECYCLED\DESKTOP.INI
D:\UNICODE\.
D:\UNICODE\..
D:\UNICODE\8-BITS
D:\UNICODE\ENTTEST.HTM
D:\UNICODE\IBMGRAPH.TX2
D:\UNICODE\TEST.BTM
D:\UNICODE\TEST2.BTM
D:\UNICODE\TEST3.BTM
D:\UNICODE\TEST4.BTM
D:\UNICODE\U0000.HTM
D:\UNICODE\U0400.HTM
D:\UNICODE\U1000.HTM
D:\UNICODE\8-BITS\.
D:\UNICODE\8-BITS\..
E:\4DOSSWAP.000
E:\4DOSSWAP.001
E:\4DOSSWAP.002
E:\RECYCLED\DESKTOP.INI
F:\RECYCLED\DESKTOP.INI
G:\K-AND-K\KNK-A10.HTM
G:\K-AND-K\KNK-A11.HTM
G:\K-AND-K\KNK-A12.HTM
G:\K-AND-K\KNK20~39.HTM
G:\K-AND-K\2005\0410.GIF
G:\K-AND-K\2005\0411.GIF
G:\K-AND-K\2005\0412.GIF
G:\RECYCLED\DESKTOP.INI
G:\TXDN\4DOSTECH.TXT
[snip pile of other stuff downloaded]

Correction to list files only and not directories ('-d' added):

alias new=`dir /[d-%1] /a-d /s /f /x c:\*.* d:\*.* e:\*.* f:\*.* g:\*.*`

 

[Norman L. DeForest]

BTW, on the subject of Recent File heuristics and malware obfuscation,
I recall that during the early development of my D-browse program that
Windows didn't allow changing a file's creation and last accessed ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dates .... only the recently modified date. Yet, all three dates can ^^^^^
be changed in plain DOS. I didn't pursue this, just made a mental
note. It may be possible using low level interrupts to bypass Windows'
protection of the two dates, I dunno. But I do know it couldn't be
done using the LFN interrupt services. And I'll add that my work is
strictly with Win 9X/ME.

It seems to me that it should be easy to change the creation date.
1. Copy the file to a temporary file.
2. Delete the original file.
3. Change the system date and/or time.
4. Read the temporary file and write to another new file with the new
creation date and time.
5. Rename the new file to have the same name as the original file to be
changed.
6. Delete the temporary file.
7. Reset the system date and/or time back to the correct setting.

Alternatively,
1. zip the file (archiving the date and time in the zip file),
2. change the date stored in the zip file,
3. delete the original file,
4. unzip the zip file.

 

[null]

not to put too fine a point on it but that's still got the same
problem... the data you're using to decide what's new or not (and
therefore what to scan) can be easily forged/modified/whatever...

As a matter of general interest, I'd like to hear of examples of
malware that have actually been successful in avoiding "recent
modiification date" heuristics. I think others would as well.

No doubt most users should do routine complete scans, just in case.

I'm going to remove the DOS av scanner - batch file aspect from my web
site.

Art

http://home.epix.net/~artnpeg

 

[null]

It seems to me that it should be easy to change the creation date.
1. Copy the file to a temporary file.
2. Delete the original file.
3. Change the system date and/or time.
4. Read the temporary file and write to another new file with the new
creation date and time.
5. Rename the new file to have the same name as the original file to be
changed.
6. Delete the temporary file.
7. Reset the system date and/or time back to the correct setting.

Alternatively,
1. zip the file (archiving the date and time in the zip file),
2. change the date stored in the zip file,
3. delete the original file,
4. unzip the zip file.

If the black hats go to that much trouble to obfuscate dates, the
white hats can counter by analyzing erased files. Then the black hats
will be forced to do a defrag as well

Art

http://home.epix.net/~artnpeg

 

[null]

On Wed, 13 Apr 2005 05:40:13 -0300, "Norman L. DeForest"

Something like this (all items less than 2 days old)? (With snippage to
reduce post size and one comment about a correction.)

<pardon my snippage as well>

Very interesting, Norman. I haven't used 4DOS at all, though I once
downloaded, installed it, and played with it a little when it went
free. Right now I don't have it on my machines.

Art

http://home.epix.net/~artnpeg