Image recovery: Recovering fragmented image files from flash memorycards

[Tonny Iversen]

Hello. I'm trying to figure out if it exists any software that might be able to recover fragmented images from flash memory cards / digital cameras if the card was quick-formatted (i.e. if the card has a zeroed out FAT), or some details about the jpeg algorithm to figure out how difficult development of such would be.

My main interest is for .jpg/.jpeg files.

In the situation when the FAT contains only zeroes, recovery of non-fragmented image files seems to be easy for many (or maybe most?/all?) file / image /data recovery software products, but I haven't been able to find any that can recover anything more than the first fragment of the file if the file is fragmented.

I don't know the specifics of jpg encoding, but maybe someone here knows if the different parts of the image part of the jpg files are numbered insome way, or contains some patterns/fields that can be used to connect later fragments together with the first fragment? If not, maybe something like FFT/Fouier transformations/frequency analysis or something similar/else could be used to do a qualified guess about which fragments belong toeach other and maybe present the user with some alternatives?

I guess that if image files are fragmented on a memory card, the chance is pretty high that the files only has two fragments. At least this is my guess if the card was new or formatted before use, then used to take pictures and erase some in between, and then quick-formatted afterwards. And if the directory entries are intact and most files are defragmented, I guess it could be possible also to do an educated guess about which clusters are fragments belonging to other files. Also, comparing the starting clusters from all files could indicate if the file is actually fragmented, if the difference between the starting cluster of file n and n+1 is less than the number of clusters necessary to store the file. And if the card was new, zero-filled or filled with a specific pattern before use, it should be obvious if a specific cluster was in use or not before the quick-formatting. (Maybe a full formatting fills the cards with a specific pattern? I haven't tried...)

Does anyone know of any software doing something like this or is anyone considering making something like this? I'm to pressed for time to consider developing something like this right now, but if someone else wants to look into this, I'd be more than happy to be involved in discussions around the development if you want (though not very much since I don't have much time available). And if someone could comment on these issues, it would be great if I or someone else should consider developing something like this.

Best regards
Tonny Iversen ([email protected])

 

[Hans-Peter Diettrich]

In the situation when the FAT contains only zeroes, recovery of
non-fragmented image files seems to be easy for many (or maybe
most?/all?) file / image /data recovery software products, but I
haven't been able to find any that can recover anything more than the
first fragment of the file if the file is fragmented.

First you'll have to find out the cluster size, then the directories,
which give you the starting cluster, name and size of all the files.

With fragmented files you'll have to combine every starting cluster with
every other (not yet assigned) cluster on the medium, a very time
consuming process. Also consider erased files, which can look like valid
files, up to the point where a cluster was overwritten later. If you
miss to detect the file as deleted, you'll spend much time in trying to
find the no longer existing continuation clusters.

I don't know the specifics of jpg encoding, but maybe someone here
knows if the different parts of the image part of the jpg files are
numbered in some way, or contains some patterns/fields that can be
used to connect later fragments together with the first fragment? If
not, maybe something like FFT/Fouier transformations/frequency
analysis or something similar/else could be used to do a qualified
guess about which fragments belong to each other and maybe present
the user with some alternatives?

This requires a "map" of all clusters, containing the characteristic
information of each cluster. I made such a map already myself, for
restoring segmented text files by manually pairing the ends and begins
of the clusters. But with image data? At least you'll need an file
checker, that can determine whether a chain of clusters *might* be a
valid jpeg file.

I guess that if image files are fragmented on a memory card, the
chance is pretty high that the files only has two fragments. At least
this is my guess if the card was new or formatted before use, then
used to take pictures and erase some in between, and then
quick-formatted afterwards. And if the directory entries are intact
and most files are defragmented, I guess it could be possible also to
do an educated guess about which clusters are fragments belonging to
other files.

FAT32 allocates files strictly sequentially, until the medium is full.
You can assume that no overlapped writes occured, to multiple files at
the same time. In this case a recovery will work quite well. When files
have been erased, and the clusters have been reused later, you'll have
much more problems.

DoDi

 

[zac.badenoff]

Does anyone know of any software doing something like this
or is anyone considering making something like this?
I'm to pressed for time to consider developing something
like this right now, but if someone else wants to look into this,
I'd be more than happy to be involved in discussions around
the development if you want (though not very much since I don't
have much time available). And if someone could comment on these
issues, it would be great if I or someone else should consider
developing something like this.
Tonny Iversen ([email protected])

Tonny,

'ere ya go matey!

It's been done so there's no need to try and develop something yourself.

http://www.ontrackdatarecovery.com/

Cheers,

__
zb

 

[Rob]

Tonny,

'ere ya go matey!

It's been done so there's no need to try and develop something yourself.

http://www.ontrackdatarecovery.com/

Cheers,

__
zb

Someone looking for ontrack recovery

found a link http://www.lemonshare.net/Software/1782.html

 

[cmyk]

FAT32 allocates files strictly sequentially, until the medium is full.
You can assume that no overlapped writes occured, to multiple files at the same time. In this case a recovery will work quite
well. When files have been erased, and the clusters have been reused later, you'll have much more problems.

DoDi

Hi Hans,

Under the FAT system, if you delete a file, as one might do with a poor shot, then it's space is simply marked as being available
for re-use. If that file happens not to have been the last one written, then there is a good chance that either the next file
written to the media, or the one after that, will be fragmented. For example (in simple terms, ignoring cluster sizes), if the
deleted file occupied 1000K and the next two files written each occupied 800K, the second of those files would be fragmented into
200K and 600K segments. Similarly, if the next file written was 1200K, it would be split into 1000K and 200K segments. Given the
variability of jpeg file sizes and the tendency of many users to delete files when they review them, fragmentation over 3 or more
clusters is quite possible.

Cheers

 

[Hans-Peter Diettrich]

Under the FAT system, if you delete a file, as one might do with a poor
shot, then it's space is simply marked as being available for re-use. If
that file happens not to have been the last one written, then there is a
good chance that either the next file written to the media, or the one
after that, will be fragmented.

FAT32 appends files until the end of the medium is reached. This is an
optimization to prevent file segmentation. Deleted files are overwritten
only after the medium is full.

DoDi

 

[Phil Carmody]

FAT32 appends files until the end of the medium is reached. This is an
optimization to prevent file segmentation. Deleted files are
overwritten only after the medium is full.

I can think of few techniques _more_ likely to cause fragmentation!
(Assuming that's what you meant by "segmentation".) Every temporary
file (just think of your browser cache etc.) will create a hole, for
example. Every hole is a place where the end of one file and the
start of another, or even worse the middle of a file, will later be
placed.

Phil

 

[cmyk]

FAT32 appends files until the end of the medium is reached. This is an optimization to prevent file segmentation. Deleted files
are overwritten only after the medium is full.

DoDi

FAT32 does not do this - that is a task for the OS or, perhaps, the media controller. Whether a particular camera or memory card
controller would do this when writing to the media is entirely up to the card or camera's maker. I know of none that have published
anything to the effect that their's behaves this way. In fact, for digital media, doing so is pointless from an optimisation
standpoint.

 

[Hans-Peter Diettrich]

I can think of few techniques _more_ likely to cause fragmentation!
(Assuming that's what you meant by "segmentation".)

Right, I meant fragmentation.

Every temporary
file (just think of your browser cache etc.) will create a hole, for
example. Every hole is a place where the end of one file and the
start of another, or even worse the middle of a file, will later be
placed.

I suppose that image files are written sequentially, one after another.
If not, the problem is very hard (NP?).

DoDi

 

[Hans-Peter Diettrich]

FAT32 does not do this - that is a task for the OS or, perhaps, the
media controller.

Read the available documentation about the FAT32 related data structures.

DoDi

 

[cmyk]

I did.

There is nothing in the FAT32 data structure that dictates the behaviour you claim.

 

[Hans-Peter Diettrich]

I did.

There is nothing in the FAT32 data structure that dictates the behaviour
you claim.

Look at FSInfo.FSI_Nxt_Free.

DoDi

 

[cmyk]

fsi.FSI_Nxt_Free specifies the lowest free cluster - there is nothing requiring this to occur after the last used cluster. It is
simply there to speed up disk writes on magnetic/optical media.

 

[Hans-Peter Diettrich]

fsi.FSI_Nxt_Free specifies the lowest free cluster - there is nothing
requiring this to occur after the last used cluster. It is simply there
to speed up disk writes on magnetic/optical media.

And consequently files are written contiguously, until the medium
becomes full.

DoDi

 

[cmyk]

And consequently files are written contiguously, until the medium
becomes full.

DoDi

Not necessarily if the lowest free cluster occurs before the last occupied cluster.

 

[Phil Carmody]

And consequently files are written contiguously, until the medium
becomes full.

Are you sticking by your previous
"""
Deleted files are
overwritten only after the medium is full.
"""
claim?

If so, consider this sequence:
1) Create A
2) Create B
3) Delete A
4) Create C
After stage 3, what is the lowest free cluster? Is it after B,
or where A was?


If not, then can you point out where you retracted the claim?
And maybe why you held that point of view in the first place.
And what made you change your mind.

Phil

 

[Hans-Peter Diettrich]

Are you sticking by your previous
"""
Deleted files are
overwritten only after the medium is full.
"""
claim?

Yes, of course :-]

If so, consider this sequence:
1) Create A
2) Create B
3) Delete A
4) Create C
After stage 3, what is the lowest free cluster? Is it after B,
or where A was?

Why bother with the lowest free cluster?

Of course it's where A has been, but that doesn't affect the write
pointer, which stays behind B.

DoDi

 

[Phil Carmody]

Are you sticking by your previous """
Deleted files are
overwritten only after the medium is full.
""" claim?

Yes, of course :-]

If so, consider this sequence:
1) Create A
2) Create B
3) Delete A
4) Create C
After stage 3, what is the lowest free cluster? Is it after B, or
where A was?

Why bother with the lowest free cluster?

Of course it's where A has been, but that doesn't affect the write
pointer, which stays behind B.

Each of the above operations is atomic, so what the **** is a write
pointer? You're barely making sense, so it's hard to argue against
you.

The FAT32 system apparently (see posts by other posters upthread)
writes each new file at the lowest cluster it can. That's from
where A has been. This is in agreement with all the prior FAT
systems that I have been familiar with.

If you can't work out how and where you're wrong given this
information, I simply don't have the time and patience to spoon
feed you any more. Perhaps try on comp.i.dont.have.a.clue ?

Phil

 

[Hans-Peter Diettrich]

Each of the above operations is atomic, so what the **** is a write
pointer? You're barely making sense, so it's hard to argue against
you.

You have read about FSInfo.FSI_Nxt_Free?

Isn't that a write pointer?

The FAT32 system apparently (see posts by other posters upthread)
writes each new file at the lowest cluster it can.

So you think that the system scans the FAT, each time a cluster has to
be allocated to an file or directory?

This is in agreement with all the prior FAT
systems that I have been familiar with.

Then familiarize with the new features of FAT32, before posting nonsense ;-]

DoDi

 

[cmyk]

Then familiarize with the new features of FAT32, before posting nonsense ;-]

And if you took the time to study how FSI_Nxt_Free is implemented, you wouldn't keep posting stuff like this. FSI_Nxt_Free points to
the lowest available cluster. Nothing more & nothing less. Nowhere is is said that there can't be any used clusters after this. Get
it?