USB flash drive

[Crouchez]

Is one of these a viable option to be a file server - what kind of
limitations are there apart read/write speed?

 

[kony]

Is one of these a viable option to be a file server - what kind of
limitations are there apart read/write speed?

Well, no, a fileserver is a computer while that's just
media... but I realize what you mean. "Generally", a
fileserver might have hard drives, but also unless it sees
only very little activity (few clients and/or small files)
the goal is to cache the files in main memory, essentially
making the network the bottleneck (unless client I/O is
substantial). I suppose the real question is why you are
choosing a USB flash drive? I don't mean to suggest there
aren't any positive attributes to it, only that it is a bit
unconventional and higher cost per GB storage... when in a
fileserver, it is generally the case that if there is a
whole system just dedicated to serving files, there's
probably a lot of files, you'd want a high storage capacity
to make it a bit futureproof even if you weren't going to
fill all drives immediately... but it can also mean you
merely have empty bay(s) to add more drives.

You could put some files on a USB flash drive if you wanted
to, plugged into a fileserver, but that seems a little
insecure in that anyone could just walk up and pull your USB
drive out and stuff it in their pocket. Read & write speed
is generally slower than a HDD too, and latency is much
higher which would have the most drastic effect reading many
small files... but per your needs, the performance could
still be acceptible. Since a USB flash drive is a handy
thing to have regardless of the fileserver, you might get
one (if you didn't have it already), and try it to see if
it's acceptible.

 

[kony]

Is one of these a viable option to be a file server - what kind of
limitations are there apart read/write speed?

The other limitation is the # of write cycles of flash
memory. SLC flash chips tend to have an order of magnitude
higher write cycles than MLC, and are faster read and write,
but also more expensive per capacity and ultimately tend to
reduce the capacity of any generation of a device per the
space required o put more chips in. We're talking about
over 10,000 write cycles with wear-leveling though, or over
100,000 w/SLC and even this number might be lower than
current tech can achieve so if you plan on this high write
activity you'll have to decide what the required lifespan of
the storage is and have an idea if your use will exceed the
write cycle limitation.

 

[Crouchez]

cheers. i was also looking at whether the device would choke under heavy
load because of the bandwidth limit - i've had usb devices not work because
there was no bandwidth to allocate - not sure how all this works.
theoretically there's 480mb/sec with usb2 - not sure whether read/write is
up to this speed though

 

[Grinder]

cheers. i was also looking at whether the device would choke under heavy
load because of the bandwidth limit - i've had usb devices not work because
there was no bandwidth to allocate - not sure how all this works.
theoretically there's 480mb/sec with usb2 - not sure whether read/write is
up to this speed though

480mb/sec is *transfer* speed across usb2. You will additionally be
subject to the read/write speeds on the far end of the pipe. Also, I
suspect 480mb/sec is some sort of theoretical maximum that no device has
ever delivered.

 

[kony]

cheers. i was also looking at whether the device would choke under heavy
load because of the bandwidth limit - i've had usb devices not work because
there was no bandwidth to allocate - not sure how all this works.
theoretically there's 480mb/sec with usb2 - not sure whether read/write is
up to this speed though

The 480mbps theory typically translates into under 35MBps in
reality on large files, or much slower on many small ones.
If your fileserver were on a 100Mb lan, the larger file read
and write performance might never matter as the lan would be
the bottleneck, but there's also the performance of the
flash *drive* device itself, whether it can (write, in
particular) handle the I/O rate the USB bus and lan allow,
or if it's a further bottleneck.

I suggest that if performance is a primary concern that you
not use a USB flash drive. Use it if you need to decrease
the chances of a hard drive crash over the long term, and
some size reduction or resistance to shock and temperature
extremes, or to have a portable device.

 

[Paul]

cheers. i was also looking at whether the device would choke under heavy
load because of the bandwidth limit - i've had usb devices not work because
there was no bandwidth to allocate - not sure how all this works.
theoretically there's 480mb/sec with usb2 - not sure whether read/write is
up to this speed though

As I understand it, there is a penalty for writing small files, versus writing
a large ZIP file to the USB flash. On reads, I don't know if it would matter as much.
Every time a file is written, a directory structure needs to be updated, as well
as the file itself being written.

The flash side of the interface has read and write limits. MLC and SLC flash
operate at different speeds. MLC is cheaper. Some flash drives are faster
than others, and may have a speed rating printed on the packaging. The very slowest
might only manage 1MB/sec, while faster ones might achieve 20MB/sec. So right there,
there is a big opporrunity for improvement (but still not getting near hard drive
performance).

The USB bus uses a polled mode protocol, whereby the host probes the
devices regularly, for work to do. Someone in one of the news groups, worked
out that the maximum transfer rate you could expect, based on this
protocol, was around 57MB/sec. (And gets slower, if there are a lot of
idle USB devices connected.)

There are other implementations of flash disks. For example, this:

Transcend 32GB Solid State Disk(SSD) 2.5” Flash Disk Model TS32GSSD25-M $430
http://www.newegg.com/Product/Product.aspx?Item=N82E16820208317

(Important keywords to look for - "Wear leveling" in the spec)
http://www.transcendusa.com/support/dlcenter/datasheet/Datasheet for SSD25.pdf

That particular device has a PATA interface. It has 44 pins, the same as a laptop
drive. To use that device on a desktop, you'd need a 44 pin to 40 pin adapter.
The adapter will have a separate short cable plus power plug, as the 44 pin laptop
standard, has pins for power, while on a desktop 40 pin interface, the power has
to be provided separately. So to experiment with that SSD, you'd need to pick up
an adapter as well. Here is a picture of one. They also come in an adapter version
which is mounted on a small piece of PCB.

http://www.mini-itx.com/store/images/25inIDEadapter.jpg

Wear leveling is important, as it helps mitigate the fact that flash devices
have limitations on the number of times they can be written per sector. This is different
than a USB flash device, which typically doesn't have wear leveling. The SSD
can last for years, while the USB flash can be worn out in a much shorter time
by the same sectors getting written all the time. Since there was a patent on
wear leveling, expect to see a limited number of manufacturers using the best
algorithms.

http://en.wikipedia.org/wiki/Wear_leveling
http://www.pat2pdf.org/patents/pat6850443.pdf

I notice in the datasheet for the Transcend product, that there is no mention
of the raw read/write bandwidth. The part numbering scheme, shows that the last
letter can be -M or -S. The latter corresponds to SLC flash, which should be
faster. So while the 32GB version looks nice from a size perspective, the
bandwidth properties of the 16GB -S unit could be significantly better.

http://www.transcendusa.com/Products/ModDetail.asp?ModNo=164

On the last page of this datasheet, Sandisk actually lists the performance of
their product.

http://www.sandisk.com/Assets/File/pdf/oem/SanDisk SSD UATA 5000 1.8.pdf

"Read 62MB/sec <---- PATA version
Write 35MB/sec"

"Read 63MB/sec <---- SATA version
Write 38MB/sec"

Since the PATA interface is capable of at least 100MB/sec, the quoted rates
are limited by flash technology and the associated controller.

Notice the number of IOPs quoted for the technology (7000 IOPs at a tiny access
size). That could significantly enhance server operation, for repetitive read info.
A SSD can seek a lot faster than conventional rotating media. Too bad the
size is so small.

Paul