Corruption?

[CWLee]

I read (on this NG, and on two other more narrowly focused, but Windows
related, NGs) about various problems people have. Often a solution is
provided that works the first time. Other times a series of things have to
be tried before the problem is solved. In some percentage of cases,
however, the answer seems to be that some file has become "corrupted" but
with no explanation of how that might happen, or how to prevent it from
happening again. Then the solution usually involves re-installing some
software.

Why/how do files become "corrupted"? Is it a software or hardware problem?
Does it happen at the nuclear level, the molecular level, or where? Is it
caused by nearby external events, like low-flying aircraft, lightning
strikes, or copulating dogs?

My friends who know much more about computers than I do all become vague and
uneasy when I ask this question of them. Anyone here have something useful
to contribute to this topic?

 

[George Hester]

Adding and removing software is the most common cause of "corruption."

 

[Steve Nielsen]

Files get corrupted in all sorts of ways, among them are power problems,
hardware problems, software problems, user error. The question is too
general and there is no single answer for it.

Steve

 

[CWLee]

Files get corrupted in all sorts of ways, among them are power problems,
hardware problems, software problems, user error. The question is too
general and there is no single answer for it.

An off-line source sent me this. Can you comment on it?

+++++++++++++++

When a computer program wants to operate on a disk file, it must go through
the operating system as a mediary. It must ask the operating system to
"open" the file. When it wants to see the contents of an existing file it
must ask the operating system to "read" the file. When it wants to modify
the contents of a file it must ask the operating system to "write" to the
file. When it is finished, it must ask the operating system to "close" the
file.

Most operating systems (windows, unix, etc.) try to "speed up" file
operations by caching. This means that when a program tells the operating
system to "write" to a disk file, the operating system will probably just
"write" to an area of memory, and make a note to iteslf that at some later
time it needs to really write that memory to the hard disk. This makes it
appear that the program is running faster because the writes finish sooner.
Have you ever been sitting at your computer, doing nothing (like talking on
the phone) and all of a sudden you hear/see activity on your hard disk (the
LED flashing)? Thats the operating system finally getting around to
writting all those blocks back to the disk. Flushing its caches.

A file can become corrupt when the operating system fails to ever perform
this write of the memory buffer back to the actual hard disk. So what makes
the operating system ever "not" do this? When the operating system looses
control of your computer. Turning off your computer without going through
the shutdown menu, or a power failure, are two popular examples. When the
operating system "freezes" (windows), and the user is forced to perform a
hard reset is another common reason. There could also be true hardware
failures. A nearby lightning strike may impact the hardware by inducing a
reset, or inducing a bit-flip in the memory. If the memory location was one
of these pending write buffers, or the flag that tells the operating system
it needs to perform a pending write, then the disk file can become
corrupted. And finally, if only an individual program "crashes", s! ome of
its file modifications may have been completed while others have not,
leaving the file contents in an inconsistent state.

Programs are not the only things that operate on files. The operating
system itself opens and uses files. Whenever it runs a program, the program
instructions themselves are stored in a file. And on every hard disk is a
directory structure that contains information telling the operating system
the exact location on a hard disk that a file's data is stored. Sometimes,
that directory information can be the file information that gets corrupted.
This is usually called a "hard drive crash". In this case, your data is
still on the disk, but the pointers to its exact location are lost. Its
like loosing your address book. You have not really lost all your friends,
you just can't contact them because you no longer know their phone number or
street address.

+++++++++++++++++

Many thanks.

 

[Steve Nielsen]

Ok.

Steve

CWLee wrote:

REM "Steve Nielsen" <[email protected] wrote
REM
REM
REM > Files get corrupted in all sorts of ways, among them are power
problems,
REM > hardware problems, software problems, user error. The question is too
REM > general and there is no single answer for it.
REM
REM
REM An off-line source sent me this. Can you comment on it?
REM
REM +++++++++++++++
REM
REM When a computer program wants to operate on a disk file, it must go
through
REM the operating system as a mediary. It must ask the operating system to
REM "open" the file. When it wants to see the contents of an existing
file it
REM must ask the operating system to "read" the file. When it wants to
modify
REM the contents of a file it must ask the operating system to "write"
to the
REM file. When it is finished, it must ask the operating system to
"close" the
REM file.
REM
REM Most operating systems (windows, unix, etc.) try to "speed up" file
REM operations by caching. This means that when a program tells the
operating
REM system to "write" to a disk file, the operating system will
probably just
REM "write" to an area of memory, and make a note to iteslf that at
some later
REM time it needs to really write that memory to the hard disk. This
makes it
REM appear that the program is running faster because the writes finish
sooner.
REM Have you ever been sitting at your computer, doing nothing (like
talking on
REM the phone) and all of a sudden you hear/see activity on your hard
disk (the
REM LED flashing)? Thats the operating system finally getting around to
REM writting all those blocks back to the disk. Flushing its caches.
REM
REM A file can become corrupt when the operating system fails to ever
perform
REM this write of the memory buffer back to the actual hard disk. So
what makes
REM the operating system ever "not" do this? When the operating system
looses
REM control of your computer. Turning off your computer without going
through
REM the shutdown menu, or a power failure, are two popular examples.
When the
REM operating system "freezes" (windows), and the user is forced to
perform a
REM hard reset is another common reason. There could also be true hardware
REM failures. A nearby lightning strike may impact the hardware by
inducing a
REM reset, or inducing a bit-flip in the memory. If the memory
location was one
REM of these pending write buffers, or the flag that tells the
operating system
REM it needs to perform a pending write, then the disk file can become
REM corrupted. And finally, if only an individual program "crashes",
s! ome of
REM its file modifications may have been completed while others have not,
REM leaving the file contents in an inconsistent state.
REM
REM Programs are not the only things that operate on files. The operating
REM system itself opens and uses files. Whenever it runs a program,
the program
REM instructions themselves are stored in a file. And on every hard
disk is a
REM directory structure that contains information telling the operating
system
REM the exact location on a hard disk that a file's data is stored.
Sometimes,
REM that directory information can be the file information that gets
corrupted.
REM This is usually called a "hard drive crash". In this case, your
data is
REM still on the disk, but the pointers to its exact location are lost.
Its
REM like loosing your address book. You have not really lost all your
friends,
REM you just can't contact them because you no longer know their phone
number or
REM street address.
REM
REM +++++++++++++++++
REM
REM Many thanks.
REM

 

[Steve Nielsen]

If disk operations are not completing yes it would result in file
corruption. The question is, why are they not completing? It's the same
answer; power problems, hardware problems, software problems and/or user
error, just as your off-line source said.

If you are experiencing file corruption then you need to determine what
is causing it. What is being done at the time? Under what conditions
does the corruption occur? What programs and processes are running? Do
you have lousey power mains? Does the hardware test good? Is the system
free of viruses? Etc.

Steve

 

[CWLee]

If you are experiencing file corruption then you need to determine what
is causing it. ...

No special or immediate problems that I can trace to file corruption on my
computer. I was just trying to do some preventive planning, since I noted
so many other people seem to have problems which are, ultimately, attributed
to file corruption. I had one incident (in 4 years) in which I lost a few
hundred emails, but the system has worked fine since then, and most of those
emails were available on my backup system. I never determined what caused
that incident, and am willing to live with the ambiguity. Thanks for your
comments, not just to me but to the various others who have sought
assistance here.