If you use FAT (win 9x), fragmentation can be a big problem. With NTFS
(XP, Vista), the impact of fragmentation is reduced somewhat but not
completely negated.
It is good practice to defrag the drive regularly - about once a week,
despite others' claims of it making no difference. If the drive is
heavily fragmented, then the defrag will take longer, and you ought to
defrag more often then so it doesnt come to that stage. For light
fragmentation, it will take only a few mins anyway, so you are not
wasting much time.
Defragmentation can have a performance impact on games or other
programs that involve frequent disk I/O. Moreover, defragmentation
prolongs the life of the drive since the R/W head has to do less work
in picking up a contiguous file compared to a fragmented file.
I personally stopped worrying about manual defrags and how often I need
to defrag since I switched to Diskeeper 2007. Its set on autodefragment
and runs in the background without bothering my gaming or other
resource intensive activities. Defrags when it needs to and gets a
chance during idle time. So far, so good.