Nina DiBoy said:
Note that OneCare passed for XP in June 06. The failure in this test was,
according to sources less than 1%.
-steve
From the VB100 site:
VB100 test procedures
A VB100 award denotes that the product in question showed, in its default mode,
100 per cent detection of In the Wild test samples and no false positives in a
selection of clean files.
For on-demand scanning of files, detection is considered to be a note in the
product log file that the file is infected or very likely so. For on-demand
scanning of boot sector viruses, a notification or log file entry is required.
For on-access scanning the matter is a little more confusing, since the best
method of testing - executing all files and using the results from this activity
- is clearly impractical. Detection is thus judged by a product denying access
to an infected file when the file is opened for writing.
For boot sector on-access scanning a visible notification or log file entry is
required. In this case denial of access is not a useful guide to detection since
the VB boot sector test floppies are all blank as far as file contents are
concerned. Since denial of access is likely to show a blank disk as the only
detectable effect, this is not particularly useful. The addition of extra files
to the disk for use in deciding whether access has been denied was decided
against, for in past testing some products were only able to detect a boot
sector virus on a floppy containing other files - a situation which would be
apparent only with the use of disks in their current state.
Products which cannot be cajoled into producing reasonable logs on demand are
checked by setting the product to delete and/or disinfect. The files are then
scanned until no more detections are present, if necessary manually noting those
files which are detected as infected but are not deleted or disinfected.
Disinfected files are removed from the test set by use of CRC checking, and
those files left in the test set are considered to be misses.
Near misses
There remains ample opportunity for products to miss detection, in our tests, of
files which they are perfectly able to detect - why? Of the many potential
answers, two are most likely. First, there are the matters of default extension
lists, a common area for failure over the years, in which products have failed
to gain VB100 awards because the default extension lists did not include
possible extensions for In the Wild viruses. In most cases these extension-based
problems are easily solved by an administrator adding extensions to the default
list. We could perform these changes prior to testing. We feel, however, that
our readers are better served if they know that they have to do this, than if we
scan all files regardless of extension.
Another example of why some products miss out on VB100 awards, is where certain
files are not scanned directly on-access. The usual assumption by the product
developers is that the files will be scanned when passed on to an application
which makes use of them. At the most common level this covers such objects as
ZIP files, which are often not scanned until unzipped and EML files, which are
not scanned until individual mails are pulled from within. From a developer's
point of view these choices make sense in that leaving objects unscanned until
use creates fewer overheads. The chance of infection on a protected machine is
not increased, since scanning will occur before code execution. Such treatment
of objects does, however lead to misses under the VB100 testing methodology.
Three chances
Each product may be tested up to three times on two different test machines.
Should any product fail to work after three attempts the testing process will be
aborted for that product.
VB100 award
A VB100 award means that a product has passed our tests, no more and no less.
The failure to attain a VB100 award is not a declaration that a product cannot
provide adequate protection in the real world if administered by a professional.
We would urge any potential customer, when looking at the VB100 record of any
software, not simply to consider passes and fails, but to read the small print
in the reviews.
