write endurance - solid state flash drives, windows xp, virtualmemory

[jameshanley39]

I am interested in booting windows xp off a solid state drive.

The 8GB+ solid state flash drives are a bit expensive for me..

I am looking at 4GB and less(2Gb,1GB)
...
The compact flash cards may not have such good endurance.. - they may
wear out.

but there are companies making what some may market as IDE Flash
Modules . .(e.g. made by Emphase, Innodisk, Transcend.)

They have write endurance of 1 to 4 million cycles, and have a
controller utilising a wear levelling algorithm.

How long will they last running windows xp?

Would I -have- to disable virtual memory / set to no page file. I have
heard that virtual memory wears these things out.. But I wonder if
with such high endurance.

I have read that capacity works for it.. A larger capacity drive will
last longer. More memory cells.
So I wonder if a 2GB drive won't last so long..

BTW, they don't come with SMART data that gives how many more writes
they have / how the disk is.. do they ?
(though I have read that they are more likely to fail due to
controller error - picking a block that is already quite worn, wearing
it out completely and making the disk unusable or unreliable. )

It would be nice if the CF was 3.0, (even though more speed may mean
it wears out quicker!). But first and foremost, I want it to be
reliable. And not priced through the roof. I don't want to be
spending more than $80 / 40UKP. The big solid state drives, like 8GB,
+, are quite expensive.

I am interested in booting different operating systems e.g. FREEBSD..
without worrying about this writing issue.. and how for each OS, to
deal with it.

 

[Boo]

I am interested in booting windows xp off a solid state drive.

I have got Acronis Disk Director booting of a CompactFlash device in an IDE
adaptor but it boots XP off a normal disk drive not directly off of the flash.
Works great.

The 8GB+ solid state flash drives are a bit expensive for me..

Well, they're advertised on the UK eBay site starting at around £10 - if that's
too much then why bother at all ? I bought a pack of 3 CF to IDE adaptors for
£5 including postage from HK, arrived in 4 days.

How long will they last running windows xp?

Would I -have- to disable virtual memory / set to no page file. I have
heard that virtual memory wears these things out.. But I wonder if
with such high endurance.

I don't know what the endurance will actually be running XP directly off of one,
disabling the page file sounds like a good idea to me.

BTW, they don't come with SMART data that gives how many more writes
they have / how the disk is.. do they ?
No.

It would be nice if the CF was 3.0, (even though more speed may mean
it wears out quicker!). But first and foremost, I want it to be
reliable. And not priced through the roof. I don't want to be
spending more than $80 / 40UKP. The big solid state drives, like 8GB,
+, are quite expensive.

Just buy a CF card and an adaptor, at least that way your expenditure is limited
even if the lifetime is short.

I am interested in booting different operating systems e.g. FREEBSD..
without worrying about this writing issue.. and how for each OS, to
deal with it.

What's your particular interest in Fkash disks may I ask ?

 

[jameshanley39]

Is there a limit on how many CF cards you can use? You can
get good performance and less concern for wear by using 3 of
them, all three using SLCflashchips (normally would then
be rated for over 10MB/s write speed) and CF4 spec (not just
"CF spec" as some list this but it's CF2.x or older, not
CF4.

<snip>

Your post was brilliant, full of great advice.. I have many comments,
but at the moment, my main is ,

Does a 133x card (20MB/s) imply CF standard of > 2.0 ?

I am a bit puzzled as to CF standards..

According to wikipedia,
wikipedia
according to the CompactFlash Association CFA

Revision 2.0 - supports up to 16 MB/s data transfer
Revision 3.0 supports up to a 66 MB/s data transfer rate, along with a
number of other features.
Revision 4.0 supports IDE Ultra DMA 133 for a maximum data transfer
rate of 133 MB/s

According to
www.logicsupply.com - who have good info in compact flash cards.

Transcend industrial compact flash card
TS1GCF100I 100x 1GB - 2 million writes,
CF 4.1, 25-40MB/s, 100x, UDMA 0-4
(transcend have crap website, takes ages to load.. logicsupply is
better!)
http://ec.transcendusa.com/product/ItemDetail.asp?ItemID=TS1GCF100I

what does 25MB-40MB actually mean? read? write? a range?

thing is, 100x is 15MB/s, so CF 2.0 would handle that. And 15MB/s is
alot less than 25-40MB/s

Even if it does 40MB/s, that is within CF 3.0
What advantage is in it being CF 4.1

can the speed ever imply anything about the compact flash
specification version number?

thanks

 

[Joel]

Your post was brilliant, full of great advice.. I have many comments,
but at the moment, my main is ,

Does a 133x card (20MB/s) imply CF standard of > 2.0 ?

I am a bit puzzled as to CF standards..

According to wikipedia,
wikipedia
according to the CompactFlash Association CFA

Revision 2.0 - supports up to 16 MB/s data transfer
Revision 3.0 supports up to a 66 MB/s data transfer rate, along with a
number of other features.
Revision 4.0 supports IDE Ultra DMA 133 for a maximum data transfer
rate of 133 MB/s

Sometime we just learn to make thing as simplest as we can understand to
be able to understand and to make life a little easier to live... unless we
really need detailed technical information to write a book or something then
that's another story.

And these are my guessing .. cuz I never care to know more to confuse me.

133X = 133 times faster than whatever the original speed they use to compare
to. Same with the speed of CD/DVD writer, the CPU speed etc.. my first CD
burner was 1X and now I believe it's around 52X or so, and it seems like the
4X DVD is faster than 4X CD-R

Revision - I just think of it as firmware, generation, version of the
hardware etc..

Me, I don't care about the thing I need not to know but the basic thing
like "size", "speed", "price" and read the feedback's to see how it compares
to others carrying the higher price tag.

 

[jameshanley39]

@yahoo.co.uk" <[email protected]>
wrote:

Put two cards in a RAID0 for OS.
Put third card as pagefile, temp files for OS, browser temp
files, application scratch space if needed.

this is excellent advice, thanks

I notice some high end (DMI supporting) compact flash to IDE adaptors
have slots for master and slave. So I can put 3, even 4 in, if 2 IDE
connectors.

of course, USB slots are more freely available and more numerous.
Since I am not booting an OS for the 3rd card. Would a USB flash drive
be OK for the "third card" ?

I guess the RAID is expensive if RAIDing 2 fast cards.. so I might do
your other suggestion of 2 cards not RAIDed, but with OSnApps on the
faster one.

In this case, the partitioning, rather, I mean, separate modules,
would be for preservation of data..

Actually, if I was RAIDing, wouldn't I need 4 cards not 3 ?
2 for the OSnApps. 1 for scratch. One for Data.
Your suggestion missed out the data one.

BTW, out of interest, it sounds like you've done this before.. Whta
was your reason.. I know you don't have a problem with fan noise, so
that's not it.

Aren't these even more expensive? Perhaps I'm not clear on
what you meant above about "8GB_ solid stateflashdrives
are a bit expensive", at first I had assumed you meant an
8GB CF card but now I think you mean an 8GB SSD instead.

CF cards are fine..

You're right, the IDE Flash modules are more expensive.. a bit more.

http://linitx.com/viewcategory.php?catid=126&pp=126

I think maybe the marketting is such that you have smaller capacity
ones called IDE Flash Modules, that prob aren't 2.5" or 3.5". And
Bigger, capacity and size, are solid disk drives. All flash drives
though, and all solid state.

I was just looking at the IDE Flash modules because they tend to have
better write endurance. But the high end CF cards, sometimes marketted
as industrial, have write endurance to match them. like 1-4 million.

I'll look at CF cards now.

I notice there are SD cards too.. still Flash. Alot cheaper than CF. I
guess they prob don't do UDMA.

Their endurance probably isn't much if any better than a CF
card using SLC chips... there are only so many chips out
there they might use.




WinXP is only an OS, we still dont' know the use of the
system and even if you said "browse internet" it can vary a
lot from one person to the next how much writing that is.

You can also set up a ramdrive, there are some out there
that support at least 1GB (I dont' recall the max but I know
1GB is possible because I have 1GB set up with one). IIRC
the one I used was mentioned and d/l linked here:

http://www.mydigitallife.info/2007/05/27/free-ramdisk-for-windows-vis...

this is fantastic..

On a related note, I also found this..
Gigabyte iRAM - It's A physical RAM disk
PCI Card, takes RAM modules.. And it seems, actually stores the data..
(dunno how it does that, without losing it)

Somebody posted that he can boot windows xp in 5 seconds with it!!!

It wouldn't have the write endurance problem either.
A bit expensive, but not bad.. One of these cards, and alot of RAM
lying about, could go a long way.

They claim limited support, just Gigabyte motherboards that use one of
a bunch of certain supported southbridges..
I haven't spoken to anybody that has one,
But I have heard , from those that don't have one, that it should
work on any motherboard with a PCI slot.. Because it is essentially a
SATA drive, that just gets its 3.3V of power from PCI, and transfers
data through a connected SATA cable.

If you were setting it up in a worst case scenario with only
one card, no ramdrive, not disabling pagefile, etc - I dont'
think anyone can really tell you because everyone seems to
think about their optimizations when they do it. I don't
remember finding anyone who has actually worn theirs out
yet, so if you were to use all possible optimizations
including 3 cards w/SLC chips of good excess capacity, a
ramdrive, and have a vaguely typical XP use, in theory you
could get a minimum of a few dozen years out of this config.




I'd consider it more a question of whether you can disable
virtual memory, whether there is enough main system memory
available including consideration of whether your particular
apps reserve a lot which can vary quite a bit. On one
system I'd disabled virtual memory on, it ran fine like that
for whatever I wanted to do, until I kept getting a
mysterious crash in gaming. Finally after fiddling with
drivers and checking overheating/etc I finally turned
virtual memory back on and it clearly resolved the problem.

Also, how much you actually use virtual memory (regardless
of how much of it is allocated, since allocation <> actually
writing that much data to it) makes quite a difference. You
might have 500MB used as shown in Task Manager but
practically none of that written with data, or you might
have quite a bit written if the system doesn't have much
physical memory. In summary, whether you have pagefile
enabled or not matters a lot less than how much physical
memory you have vs the jobs ran.

ok.. so for win xp. "PF Usage" in task manager..performance, would be
of practical relevance to that - if I wanted to monitor it a bit.

Besides the XPe option. And the windows xp for legacy pcs option you
mentioned.

You mentioned earlier about moving the location of the temporary
folder used by applications.. on a per application basis. I found the
program "Process Monitor"(a combination of those old sysinternals
programs, filemon and regmon). It intercepts all registry changes and
file writes. It's prob clearer though. I can filter by paths
containing C:\ or whatever drive, and thus see all the file writes.
Regarding moving the Windows "Temp" directory.. (I think a good idea
is to have %TMP% and %TEMP% pointing to te same place)..
I find IE writes not just to
c:\docu..\user\Local Settings\"Temporary Internet Files",
but within Local Settings, it also writes to LS\Cookies, and some,
maybe IE too, writes to "LS\Application Data".
It may be better to just move the whole Documents And Settings
directory..
I found one or two links on that.
http://www.tech-recipes.com/windows_tips1409.html


Of course.. it seems with XPe at least, I won't have to work on an
application basis.. Any writes go to HDD gets intercepted and
redirected - it seems - and go to RAM.

It might also be possible to get a write filter like the one on XPe,
onto Win XP Pro. I think I saw it mentioned on this forum
http://www.mp3car.com/vbulletin/mp3car-technical/

I would do as mentioned above, leave virtual memory enabled
but set to use a different CF card than the one upon which
OS is installed.

That would wear out fast..
But
I guess it's important that that should be CF3.0/4.0 So, UDMA too?
'cos Windows would be accessing that alot..

So I guess that doesn't save on price.. It just preserves the card
holding my data.

Possibly. Consider the scenario where you might have 1.5GB
of space taken by the OS (though with XP, that would be a
pretty lean installation) and a 2GB card. Ignoring loss due
to formatting and definitions of a GB varying between binary
and decimal, let's say you have 512MB left empty. Now let's
say you wrote 1GB/day to this drive, it means you wrote
every area twice if the wear leveling was perfect.

Now contrast that with a 4GB card, which would have approx
2560MB free. Writing 1GB/day to this one you have written
to each about 0.4 times. 2/0.4 = 5, the 4GB card would last
5 times as long.




If an area is unusable the OS should mark it as bad if the
controller didn't itself. I don't recall the exact
capabilities of the controller and don't recall ever seeing
any way to get statistics, I don't think Smart is supported.




No, it wears out much slower if more speed is due to being
SLC chipped or wears about same speed if not since you're
still writing same amount per use.

It seems that the ones with good endurance, are transcend, emphase,
innodisk..
where they advertise the endurance.. and ideally, the version of spec.

sandisk - prob because of their end user market- do not advertise it.
So even newegg do not list it in their specification. But I notice the
prices of their CF cards (not their SD cards).. Are the same as the
big names..

Approximately $12.5 per GB. $25 for 2GB. 50 for 4GB ..

funnily enough, sandisk forum don't know http://communities.sandisk.com/
"or" better http://communities.sandisk.com/sandisk/ , and their
telephone technical support don't know the write endurance or CF
spec.

I
We don't know how much capacity you need, nor the prices
there. With that budget I would look for at least two 4GB CF
cards.

That would be fine.

Kind of an open end issue, all you can do is look at where
it writes a lot and see if you can designate a different
location of not being able to turn it off.

You may be overly concerned about the write cycle
limitation, even if you used low endflashcards with the
least # of write cycles possible you can still get 16GB or
more offlashmemory for $80... at least in the US they've
dropped to about $60 and up for the slower ones, but I feel
those might be a little too slow for many uses and would
suggest you get at least two 2-4GB cards with SLC chips and
CF4 spec. It might be a little over the budget but IMO the
best thing that comes close in the states would be a couple
of these:http://www.newegg.com/Product/Product.aspx?Item=N82E16820208418

For a little less but half the capacity and a speed
reduction,http://www.newegg.com/Product/Product.aspx?Item=N82E16820208296

nice.. I notice they are transcend also.. 266x and 300x. 2GB and 4GB

newegg are very good, showing the spec

http://www.newegg.com/Product/Product.aspx?Item=N82E16820208418
newegg..418
TS4GCF300 300x 4GB Transcend. $52.99
CompactFlash 4.0 compliant, UDMA 0-5, default 5.

http://www.newegg.com/Product/Product.aspx?Item=N82E16820208296
newegg....296
TS2GCF266 266x 2GB Transcend $24.49
CF 4.0 supports PIO mode 6, UDMA mode 4

http://www.newegg.com/Product/Product.aspx?Item=N82E16820208297
TS4GCF266 266x 4GB Transcend $44.99
CF 4.0 supports PIO mode 6, UDMA mode 4


amazon sell some cheap transcend. I notice they are SD - secure
digital though. Not CF.
And no doubt no UDMA.

TS4GSDHC6 4GB
10UKP inc PnP (like $20)

TS2GSDC 2GB
5UKP inc PnP (like $10)

If you're sure you don't want to RAID a couple, you might
get one of each to come in around $80, putting OS & apps on
the faster one.

There's also a hack for XP where you can set up a write
filter from XPe so reduce or eliminate writes to the
HDD/CF-as-HDD. Google may find more info on that. Windows
For Legacy PCs seems to be a distant relative inbetween XP
and XPe (more like XPe but with most basic PC user features
enabled?) that might also allow turning off some features
that cause HDD writes, but I expect that if you use decent
SLC chippedflashcards of ample capacity that you won't
have it worn out in a few months of regular use, it should
be a few years at least, if the system isn't totally retired
by then.

yeah.. I guess if using flash, dealing with this on an OS by OS basis,
I may as well.

are there laptops shipped with SSD drives using windows xp with
virtual memory and eating the drives? The average end user would not
have done any of these optimisations to protect the drive.


many thanks

 

[jameshanley39]

Revision is very important for the intended use as a
replacement for  a hard drive on a Compact Flash - IDE
adapter because it determines whether the card will operate
in UDMA mode.  Some cards rated for 150X (which is about
22MB/s) can't even exceed 6MB/s when connected to a CF-IDE
adapter in PIO mode, and use quite a lot of CPU time to do
it while UDMA mode doesn't.  It makes a night and day
difference in the performance, changing an extremely
sluggish system into one that's fairly snappy so long as the
file transfers aren't very large or huge numbers of tiny
files.

I see how PIO is bad because of its CPU utilization. But in terms of
speed.. UDMA is generally faster. But there are fast PIO and slow
UDMA..
Howcome you keep referring to PIO as 6MB/s. Why wouldn't PIO cards go
faster, like 16.7MB/s. Is it because you are saying the cheap cards
are PIO mode 1.
I suppose ATA(ATA/IDE connector) doesn't support Modes 5,6. Since that
is specific to CF. But it would support Mode 4 PIO I would think.
16.7MB/s

Another oddity is that sandisk Extreme IV cards say UDMA mode 4. But
have a max R/W of up to 40MB/s (266x), I guess they adverise as 266x,
so it's a max. The oddity though, is they may as well be on UDMA mode
3. !!
And if that's the max speed, I wonder what the minimum be like

ref-

PIO
Mode 6 - 25MB/s <--- CF2
Mode 5 - 20MB/s <-- CF2
Mode 4 - 16.7MB/s <-ATA
Mode 3 – 11.1MB/s <-ATA
Mode 2 – 8.3MB/s <-ATA
Mode 1 – 5.2MB/s <-ATA
Mode 0 - 3.3MB/s <-- ATA

UDMA
Mode 5 - 100MB/s <-- ATA/ATAPI
Mode 4 - 66.7 MB/s <-- ATA/ATAPI
Mode 3 – 44.4MB/s <-- ATA/ATAPI
Mode 2 – 33.3MB/s <-- ATA/ATAPI
Mode 1 - 25MB/s <-- ATA/ATAPI
Mode 0 - 16.7MB/s <-- ATA/ATAPI

http://en.wikipedia.org/wiki/Programmed_input/output
(better than pcguide PIO link. which is missing mode 0 and modes 5,6

http://www.pcguide.com/ref/hdd/if/ide/modes_UDMA.htm
pcguide UDMA link.

 

[jameshanley39]

There's no UDMA (0) slower than the fastest PIO.  Mainly the
point is, AFAIK nobody would pair up the low end cheap flash
chips that are slow, with the premium UDMA capable
controller, though they'll be more likely to pair the
fastest chips up with the premium controller.  This also
tends to be true because of customer demand, higher
performance more sophisticated equipment would be the most
likely to use UDMA mode instead of a camera, though I
suppose these days that generalization is changing, a lot of
purpose built ICs do far more than they used to.

Generally we can ignore all these issues and just remember
one thing - you absolutely must have a card with CF3 or 4
spec operating in UDMA mode if you want it more than
sluggish.  

Keep in mind, any modern flash chips can achieve over 27MB/s
read speed, including the low-end, if only they have the
right controller and for your purposes that means it can't
operate in PIO mode even if what I was about to write next
wasn't true.


I'm saying that I didn't care  enough to find out why they
only achieved 6MB/s, apparently they only supported a lower
PIO mode - on several different motherboards so it would
tend to implicate the cards.  These were mostly 512MB to 2GB
cards I had leftover from an old camera.


I don't understand why you are spending time pondering the
very cards you don't want to use if performance is
important.    It wouldn't be surprising, in the low end
cards, for the manufacturer to only support the bare minimum
it takes to qualify as a "Compact Flash" card, particularly
since many cameras aren't using them in PIO mode.  That
could be seen as a good thing, as today you can get a 2GB
low end card almost for nothing, $15 or less generally, but
it might be the equivalent of comparing a zip drive to a
hard drive when it comes to performance running a large(r)
OS from it on an ATA controller.  Even Win98 using Win98
shell a la 98Lite, an installation taking under 200MB drive
space and about a couple dozen MB memory, is a bit sluggish
running from a low end CF card in PIO mode.




This is the same as with any other drive, you want the
interface bandwidth to be as high as possible - modern spec,
no matter whether the actual performance could've been done
with some slower interface.  It wold serve no useful purpose
for it to be CF3 instead, CF4 is the modern spec and thus
any modern card pretending to be high performance should
adhere to that spec.

I'm going to back up and say again what you seemed to miss.
Don't think about these things.  Think about only:

CF4 spec
SLC chips
Max read speed the budget can afford
(in this order of importance)









You have no guarantee about any mode any card uses unless it
clearly specifies it.  However, what we can assume is that
something that makes a CF card more valuable is something
the manufacturer will not overlook mentioning, ie - if it is
CF3 or 4 spec. and the inherant support those mandate.
Beyond that, your choices are really quite limited since you
want not to spend a lot, with the higher cost being the
primary thing keeping many from moving to flash based
storage.-

thanks.. v good/useful discussion.