recommendations for HD video editing

[Guest]

A friend of mine who is in the video business has asked me about building a
computer to allow him to move up to high definition NLE video editing. He
is looking at using an Adobe software package under a professional version
of Windows XP (not Vista). I'm presently looking at designing a high-end
system (x2) for myself to run Linux distribution building. I am wondering
if the same system I would choose for myself would be suitable for his use.

Currently the setup I am looking at would have:
Tyan S2927A2NRF mainboard (ATX form factor)
2x AMD Opeteron Socket F dual core 3.0 GHz
16GB DDR-2 667 ECC RAM
SATA drives to suit

One concern, since my area is Linux is whether XP, in professional versions,
can handle more than 4GB. Can it go above 4GB even in a 32-bit version?

If it is possible for both of us to use essentially the same system design,
then he could potentially use my extra machine in case he has any hardware
failures.

Would you go a different route for professional HD video editing?

Any recommended custom system integrator (must know both Linux and Windows)?
I can build the machines myself, but I'm not sure I really want to this time
around.

 

[Wolf Kirchmeir]

A friend of mine who is in the video business has asked me about building a
computer to allow him to move up to high definition NLE video editing. He
is looking at using an Adobe software package under a professional version
of Windows XP (not Vista). I'm presently looking at designing a high-end
system (x2) for myself to run Linux distribution building. I am wondering
if the same system I would choose for myself would be suitable for his use.

Currently the setup I am looking at would have:
Tyan S2927A2NRF mainboard (ATX form factor)
2x AMD Opeteron Socket F dual core 3.0 GHz
16GB DDR-2 667 ECC RAM
SATA drives to suit

One concern, since my area is Linux is whether XP, in professional versions,
can handle more than 4GB. Can it go above 4GB even in a 32-bit version?

If it is possible for both of us to use essentially the same system design,
then he could potentially use my extra machine in case he has any hardware
failures.

Would you go a different route for professional HD video editing?

Any recommended custom system integrator (must know both Linux and Windows)?
I can build the machines myself, but I'm not sure I really want to this time
around.

I would advise him to buy the biggest, fastest Mac available, complete
with software. The local Apple store will be happy to help out, and will
provide an amazing amount of advice. I am not a Mac fan, but this is one
intended use where it's the simplest solution. Probably won't cost much
more than a custom-built XP machine. Besides, MAC OS-X is a version of
Linux, so you know it's a leaner, faster OS than XP.

But if you do want to build or specify an HD editing rig running on
Windows XP, you'll need a rather large and fast video card with lots of
independent video RAM. On-board video will not handle the load that HD
editing will put on it. (The video has to be rendered more or less in
real time to make editing workable, right? That's where the video card
comes in.)

A large video card entails a large case, a large PSU, efficient cooling,
and so on and so forth. My take is that your rig as described will not
handle HD editing at anything resembling a reasonable speed.

HTH

 

[kony]

I would advise him to buy the biggest, fastest Mac available, complete
with software. The local Apple store will be happy to help out, and will
provide an amazing amount of advice. I am not a Mac fan, but this is one
intended use where it's the simplest solution. Probably won't cost much
more than a custom-built XP machine. Besides, MAC OS-X is a version of
Linux, so you know it's a leaner, faster OS than XP.

It will cost substantially more for the same performance, if
you can even get as much performance.

But if you do want to build or specify an HD editing rig running on
Windows XP, you'll need a rather large and fast video card with lots of
independent video RAM.

No, video card memory speed and quantity have no benefit. A
card whose GPU has hardware decoding will help but this has
nothing to do with the memory variances between such capable
cards.

On-board video will not handle the load that HD
editing will put on it.

Yes it will do fine, except that onboard video uses a bit of
the system memory throughput for displaying video, leaving a
little less for the editing application.

(The video has to be rendered more or less in
real time to make editing workable, right? That's where the video card
comes in.)

In most cases, the CPU renders edited video. However,
instead of the video card you can also get video editing
cards to take over some of the processing.

 

[Paul]

A friend of mine who is in the video business has asked me about building a
computer to allow him to move up to high definition NLE video editing. He
is looking at using an Adobe software package under a professional version
of Windows XP (not Vista). I'm presently looking at designing a high-end
system (x2) for myself to run Linux distribution building. I am wondering
if the same system I would choose for myself would be suitable for his use.

Currently the setup I am looking at would have:
Tyan S2927A2NRF mainboard (ATX form factor)
2x AMD Opeteron Socket F dual core 3.0 GHz
16GB DDR-2 667 ECC RAM
SATA drives to suit

One concern, since my area is Linux is whether XP, in professional versions,
can handle more than 4GB. Can it go above 4GB even in a 32-bit version?

If it is possible for both of us to use essentially the same system design,
then he could potentially use my extra machine in case he has any hardware
failures.

Would you go a different route for professional HD video editing?

Any recommended custom system integrator (must know both Linux and Windows)?
I can build the machines myself, but I'm not sure I really want to this time
around.

I would, at the very least, try to find representative benchmarks for
the platforms you are thinking about.

Here, Adobe Premiere Pro 2 is tested. FX74 consists of two dual core Socket 1207
AMD processors running at 3GHz (in an ASUS L1N64-SLI WS). The QX6700 isn't even
the fastest processor in the Intel lineup. The QX6850, for example, runs at 3GHz
(as does Xeon 5365, which is still not listed on the Intel site, but is available
in Apple MacPro).

http://www.xbitlabs.com/articles/cpu/display/amd-quad-fx_10.html

http://products.amd.com/en-us/DesktopCPUDetail.aspx?id=19 (FX-74 dual)

QX6700 2.66GHz Quad/FSB1066/8MB L2 130W
http://processorfinder.intel.com/details.aspx?sSpec=SL9UL

QX6850 3GHz Quad/FSB1333/8MB L2 130W (should be similar to Xeon 5365 Socket 771)
http://processorfinder.intel.com/details.aspx?sSpec=SLAFN

For a table of OSes versus memory support, try this document, section 2.3.
There are other subtle issues with regard to OS choices, so this is not
the only consideration. How the cores are supported with the various
OSes, might be important. For example, Vista is NUMA aware, while WinXP
isn't (whatever that means).

http://dlsvr01.asus.com/pub/ASUS/mb/4GB_Rev1.pdf

Now, the next question, would be the memory thing. Does the editing
environment actually suck 16GB of HD content into memory and work
on it ? Is the excess RAM being used as a RAM disk ? Again, I would
have to see a benchmark, where 1GB, 2GB, 4GB, 8GB... were tried and
the improvements in performance noted, before I would go there. The
thing is, some motherboard and platform choices, allow a higher
memory bandwidth system to be built, so there has to be a good
reason to use a slower, larger memory array. (For example, DDR3-1333
is available for desktop systems.)

On a motherboard with FBDIMMs, there could be four memory channels.
In that case, you'd want to put at least one DIMM per channel, to
give the best memory bandwidth. That would suggest a minimum memory
size for that architecture. Similarly, if you have a multi-socket
AMD system, you'd want to install at least two DIMMs per socket,
so that every memory controller runs in dual channel mode.

Then, you study what happens as more memory is added. On an FBDIMM
system, the second DIMM installed in a channel, sees longer latency,
due to the serial interconnect of devices in the channel. The FBDIMM
channel is bidirectional, which means more than one transaction
can be in flight at a time, but the speed might not be the same as
a more "raw" format such as DDR2 unbuffered or DDR3 unbuffered.

A video editing system, uses disk arrays to hold the content which
does not fit into memory. A hardware RAID gives the best possible
performance, by not burdening the processor with data movement.
(Data is DMA transferred directly into memory.) If the hardware
RAID can't meet the performance of the processor (or accelerator)
based video processing, then the system will suffer because of it.

This card, for example, takes (16) SATA disks, connected to four
adapter cables inside the PC case. You could build two eight drive
RAID-0 arrays, each would have a bandwidth of more than 480MB/sec.
The PCI Express x8 connector has a 2GB/sec per direction bandwidth, so
the bus is no longer a bottleneck. You buy a motherboard with
two large PCI Express connectors (i.e. video card slots), one takes
the video card and the other takes the Areca. Note *very carefully*
OS support driver wise, as not every OS may be supported well. Also
note that it says this card supports arrays larger than 2TB - with
today's disks this is an important factor. One way to beat the 2TB
limit on some OSes, is to declare the sector size to be larger than
512 bytes. You may want to consult a downloadable manual, if you can
find one, as to exactly how the Areca product does this. Some arrays
rely on Windows "dynamic disk" to beat the limit.

If you use RAID 0, this should put a light load on the onboard IOP on the
Areca card, and you don't want the IOP to mess up the peak performance.
If you set the thing up for RAID5, that would spoil it.

Areca ARC-1261ML-KIT PCI-Express x8 SATA II Controller Card w/Cables RAID level 0... $1070
http://www.newegg.com/Product/Product.aspx?Item=N82E16816151015

At the end of every work day, you transfer the work in progress, off
the RAID0 arrays, and onto a set of backup 1TB disks. RAID 0 is
not a reliable RAID type, and is built for speed. So backup the
data frequently, like once a day.

This forum is a good place to go, for experiences with large or
fast arrays.

http://www.2cpu.com/forums/showthread.php?threadid=82932

To search the forum, if you are a non-member like I am,
use this search engine, set the domain to "2cpu.com" and
enter your search terms at the top.

http://www.altavista.com/web/adv

Somehow, you have to gather information, about what the Adobe application
will use of the hardware, whether it is gobs of memory, perhaps
using programmable shaders in the GPU for render acceleration, or
whatever. Without a finely detailed understanding of what is going on,
you end up being nothing more than "an assembler with a screwdriver",
with a more expensive end product than is necessary. The reason
I'm talking this way, is I've seen one too many web sites today,
selling "big" hardware to video editing punters, with no other
objective than to milk the purchaser. A lot like selling you a
Cadillac, when all you wanted was "a ride".

Good luck,
Paul

 

[Ken Maltby]

A friend of mine who is in the video business has asked me about building a
computer to allow him to move up to high definition NLE video editing. He
is looking at using an Adobe software package under a professional version
of Windows XP (not Vista). I'm presently looking at designing a high-end
system (x2) for myself to run Linux distribution building. I am wondering
if the same system I would choose for myself would be suitable for his
use.

Currently the setup I am looking at would have:
Tyan S2927A2NRF mainboard (ATX form factor)
2x AMD Opeteron Socket F dual core 3.0 GHz
16GB DDR-2 667 ECC RAM
SATA drives to suit

One concern, since my area is Linux is whether XP, in professional
versions,
can handle more than 4GB. Can it go above 4GB even in a 32-bit version?

If it is possible for both of us to use essentially the same system
design,
then he could potentially use my extra machine in case he has any hardware
failures.

Would you go a different route for professional HD video editing?

Any recommended custom system integrator (must know both Linux and
Windows)?
I can build the machines myself, but I'm not sure I really want to this
time
around.

--

You might want to post this over at "rec.video.desktop"
and they can also direct you to the active pro video NGs.

Luck;
Ken

 

[Paul]

A friend of mine who is in the video business has asked me about building a
computer to allow him to move up to high definition NLE video editing. He
is looking at using an Adobe software package under a professional version
of Windows XP (not Vista). I'm presently looking at designing a high-end
system (x2) for myself to run Linux distribution building. I am wondering
if the same system I would choose for myself would be suitable for his use.

Currently the setup I am looking at would have:
Tyan S2927A2NRF mainboard (ATX form factor)
2x AMD Opeteron Socket F dual core 3.0 GHz
16GB DDR-2 667 ECC RAM
SATA drives to suit

One concern, since my area is Linux is whether XP, in professional versions,
can handle more than 4GB. Can it go above 4GB even in a 32-bit version?

If it is possible for both of us to use essentially the same system design,
then he could potentially use my extra machine in case he has any hardware
failures.

Would you go a different route for professional HD video editing?

Any recommended custom system integrator (must know both Linux and Windows)?
I can build the machines myself, but I'm not sure I really want to this time
around.

They did a test drive here, 05/31/06, using a Dell Precision 670 workstation.
Two 3.6GHz single core Xeon processors, FSB800, 2MB L2, 3GB RAM, Quadro FX3450
video card. They were able to stack up 7 layers of picture-in-picture effects.

http://premiere.digitalmedianet.com/articles/viewarticle.jsp?id=44921

System requirements and video card options are listed here.
"2GB of RAM for HDV and HD".

http://www.adobe.com/products/premiere/systemreqs/
http://www.adobe.com/products/premi...supportSearchVar=0&version=CS3&device=graphic

That article also mentions the names of some accelerator cards to research.

Paul

 

[Guest]

| Here, Adobe Premiere Pro 2 is tested. FX74 consists of two dual core Socket 1207
| AMD processors running at 3GHz (in an ASUS L1N64-SLI WS). The QX6700 isn't even
| the fastest processor in the Intel lineup. The QX6850, for example, runs at 3GHz
| (as does Xeon 5365, which is still not listed on the Intel site, but is available
| in Apple MacPro).

Turns out Adobe Premiere CS 3 (the current product) is a 32-bit app
and runs in 32-bit mode even if a 64-bit OS is used. It cannot access
more than 3GB of memory even if the machine is populated with 64GB
(which 64-bit Windows could support).

So it looks like 4GB of RAM is the max (and even then XP will not
use more than 3.?? GB of it due to the need for memory holes when
in 32-bit mode).


| Now, the next question, would be the memory thing. Does the editing
| environment actually suck 16GB of HD content into memory and work
| on it ? Is the excess RAM being used as a RAM disk ? Again, I would

Apparently not.

| have to see a benchmark, where 1GB, 2GB, 4GB, 8GB... were tried and
| the improvements in performance noted, before I would go there. The
| thing is, some motherboard and platform choices, allow a higher
| memory bandwidth system to be built, so there has to be a good
| reason to use a slower, larger memory array. (For example, DDR3-1333
| is available for desktop systems.)

Given the discovered 3GB limit (4GB practical limit in terms of
what to physically put in), I might guess a focus memory speed
would help. And I have not seen DD3 in ECC, yet.


| On a motherboard with FBDIMMs, there could be four memory channels.
| In that case, you'd want to put at least one DIMM per channel, to
| give the best memory bandwidth. That would suggest a minimum memory
| size for that architecture. Similarly, if you have a multi-socket
| AMD system, you'd want to install at least two DIMMs per socket,
| so that every memory controller runs in dual channel mode.

I've been looking at the Tyan S2927 board. Any better suggestion?


| Then, you study what happens as more memory is added. On an FBDIMM
| system, the second DIMM installed in a channel, sees longer latency,
| due to the serial interconnect of devices in the channel. The FBDIMM
| channel is bidirectional, which means more than one transaction
| can be in flight at a time, but the speed might not be the same as
| a more "raw" format such as DDR2 unbuffered or DDR3 unbuffered.

Unfortunately, I don't have the study opportunity. I need to pick
the right parts and then that is what we are stuck with.


| A video editing system, uses disk arrays to hold the content which
| does not fit into memory. A hardware RAID gives the best possible
| performance, by not burdening the processor with data movement.
| (Data is DMA transferred directly into memory.) If the hardware
| RAID can't meet the performance of the processor (or accelerator)
| based video processing, then the system will suffer because of it.

Modern SATA hard drives do DMA transfers at full speed, anyway.
RAID is still apparently a complication for Adobe's activation
software that profiles the system via hard drive serial numbers.
It used to not even work at all.

Additionally, given high volumes of data movement, any burst speed
capability in the short term may have little or no effect since
many times RAM size will be moving around. The need will be more
for aggregate total throughput (as long as there is no blockage in
processing the video).


| This card, for example, takes (16) SATA disks, connected to four
| adapter cables inside the PC case. You could build two eight drive
| RAID-0 arrays, each would have a bandwidth of more than 480MB/sec.
| The PCI Express x8 connector has a 2GB/sec per direction bandwidth, so
| the bus is no longer a bottleneck. You buy a motherboard with
| two large PCI Express connectors (i.e. video card slots), one takes
| the video card and the other takes the Areca. Note *very carefully*
| OS support driver wise, as not every OS may be supported well. Also
| note that it says this card supports arrays larger than 2TB - with
| today's disks this is an important factor. One way to beat the 2TB
| limit on some OSes, is to declare the sector size to be larger than
| 512 bytes. You may want to consult a downloadable manual, if you can
| find one, as to exactly how the Areca product does this. Some arrays
| rely on Windows "dynamic disk" to beat the limit.

The SATA controller is on the motherboard I am looking at (Tyan S2927).
Hopefully that will be sufficient speed. If not, I know of a PCIex16
SATA controller with 8 ports.


| If you use RAID 0, this should put a light load on the onboard IOP on the
| Areca card, and you don't want the IOP to mess up the peak performance.
| If you set the thing up for RAID5, that would spoil it.
|
| Areca ARC-1261ML-KIT PCI-Express x8 SATA II Controller Card w/Cables RAID level 0... $1070
| http://www.newegg.com/Product/Product.aspx?Item=N82E16816151015
|
| At the end of every work day, you transfer the work in progress, off
| the RAID0 arrays, and onto a set of backup 1TB disks. RAID 0 is
| not a reliable RAID type, and is built for speed. So backup the
| data frequently, like once a day.

I agree, RAID0 is the way to go. This isn't soem database that has to
sit on the same data for days, weeks, months, and years. Video is
data that gets shuffled around ... so shuffling it off to external
backup storage is smart.


| This forum is a good place to go, for experiences with large or
| fast arrays.
|
| http://www.2cpu.com/forums/showthread.php?threadid=82932
|
| To search the forum, if you are a non-member like I am,
| use this search engine, set the domain to "2cpu.com" and
| enter your search terms at the top.
|
| http://www.altavista.com/web/adv
|
| Somehow, you have to gather information, about what the Adobe application
| will use of the hardware, whether it is gobs of memory, perhaps
| using programmable shaders in the GPU for render acceleration, or
| whatever. Without a finely detailed understanding of what is going on,
| you end up being nothing more than "an assembler with a screwdriver",
| with a more expensive end product than is necessary. The reason
| I'm talking this way, is I've seen one too many web sites today,
| selling "big" hardware to video editing punters, with no other
| objective than to milk the purchaser. A lot like selling you a
| Cadillac, when all you wanted was "a ride".
|
| Good luck,
| Paul

 

[Paul]

I've been looking at the Tyan S2927 board. Any better suggestion?

Do you mean this one, the S2927 ? (It appears the rev 2 adds IPMI for
server management.) It does have ECC. On the Intel desktop
side, to do ECC, you'd need something like a 975X based board. The next
great thing for desktops will be Intel X38 chipset, delayed until Oct.11,
and supposed to support ECC. X38 officially supports FSB1333 for LGA775,
while 975X has unofficial support only on some manufacturer's motherboard
implementations. The advantage of the Tyan board, would be if you could
plug quad core processors into each socket. Then perhaps it would beat a
single Intel quad core.

ftp://ftp.tyan.com/datasheets/d_s2927_200.pdf

Modern SATA hard drives do DMA transfers at full speed, anyway.
RAID is still apparently a complication for Adobe's activation
software that profiles the system via hard drive serial numbers.
It used to not even work at all.

I guess whether a RAID makes sense or not, really depends on whether
everything on disk is in compressed form. In compressed form, the
data rates would not be nearly as great. Then, either the processor
or the GPU programmable shaders, have to uncompress things on the fly.
I was going on a comment from someone who was working with 150MB/sec
raw format data, and perhaps people don't work that way for video.

http://premiere.digitalmedianet.com/articles/viewarticle.jsp?id=44921

Paul

 

[Guest]

| (e-mail address removed) wrote:
|
| > I've been looking at the Tyan S2927 board. Any better suggestion?
|
| Do you mean this one, the S2927 ? (It appears the rev 2 adds IPMI for
| server management.) It does have ECC. On the Intel desktop
| side, to do ECC, you'd need something like a 975X based board. The next
| great thing for desktops will be Intel X38 chipset, delayed until Oct.11,
| and supposed to support ECC. X38 officially supports FSB1333 for LGA775,
| while 975X has unofficial support only on some manufacturer's motherboard
| implementations. The advantage of the Tyan board, would be if you could
| plug quad core processors into each socket. Then perhaps it would beat a
| single Intel quad core.
|
| ftp://ftp.tyan.com/datasheets/d_s2927_200.pdf
|
| > Modern SATA hard drives do DMA transfers at full speed, anyway.
| > RAID is still apparently a complication for Adobe's activation
| > software that profiles the system via hard drive serial numbers.
| > It used to not even work at all.
|
| I guess whether a RAID makes sense or not, really depends on whether
| everything on disk is in compressed form. In compressed form, the
| data rates would not be nearly as great. Then, either the processor
| or the GPU programmable shaders, have to uncompress things on the fly.
| I was going on a comment from someone who was working with 150MB/sec
| raw format data, and perhaps people don't work that way for video.
|
| http://premiere.digitalmedianet.com/articles/viewarticle.jsp?id=44921

I don't know what format Adobe Premiere CS 3 will store it. But I do
know that HDV compresses better than DV through use of group-of-pictures
being greater than 1 (I think it is 5 or 6). DV stores every frame in
exactly the same number of bytes. It may be the case that it stores the
video in the same format ingested. Or maybe not. But anything other
than raw uncompressed video will add to CPU work to process. And raw
uncompressed video means a lot of I/O work as an alternative. Complex
multi frame edits may require substantially more than real time to render.
Playout to HDV may be time critical, but once rendered, should be just
fine. Producing a BR-DVD should be straight forward even if it takes
more than an hour for a one hour video.

 

[opp]

I would advise him to buy the biggest, fastest Mac available, complete
with software. The local Apple store will be happy to help out, and will
provide an amazing amount of advice. I am not a Mac fan, but this is one
intended use where it's the simplest solution. Probably won't cost much
more than a custom-built XP machine. Besides, MAC OS-X is a version of
Linux, so you know it's a leaner, faster OS than XP.

But if you do want to build or specify an HD editing rig running on
Windows XP, you'll need a rather large and fast video card with lots of
independent video RAM. On-board video will not handle the load that HD
editing will put on it. (The video has to be rendered more or less in
real time to make editing workable, right? That's where the video card
comes in.)

A large video card entails a large case, a large PSU, efficient cooling,
and so on and so forth. My take is that your rig as described will not
handle HD editing at anything resembling a reasonable speed.

HTH

That is a bullshit response. On the Intel based mac hardware, XP runs
faster than OSX does as do most relevant applications (photoshop and
adobe premiere) according to Maximum PC. About your question, 32 bit
xp handles up to 4gb of RAM and 64 bit handles 8gb if I remember
correctly. Furthermore, most turnkey editing systems such as AVID
don't have as much RAM as you're describing. I'm really not sure about
the necessity of that much outside of database or webservers. My
recommendation is to built a smaller quieter box (with lower heat and
power consumption!!!) and get a hardware HD decoder such as an AJA
card http://www.aja.com/html/products_windows_xena_2K.html .

Forget about specs first and find out what system your friend wants to
edit on and the type of application he's doing. If he wants Final Cut
HD, mac is really his only option and I'd still recommend an AJA
(check out the IO model). If he wants AVID, Vegas, or Premiere Pro, he
needs a PC. Personal preference and learning curves are far bigger
obstacles than saving 12 seconds in a render.

If he prefers a PC based editing solution, I'd really recommend a much
smaller amount of ram, maybe 2-4gb of dual channel ddr2 in a faster
clock speed and any of the newer single chip, dual or quad core chips
should do fine. Having a hardware decoder such as the AJA will be far
more valuable than a ridiculous processor configuration. There's no
reason to reinvent the wheel since most turnkey editing systems don't
have the kind of specs you listed.

The most important thing to remember is regardless of your OS,
processor, or ram configuration, the biggest factor in hardware will
be your HD configuration. I recommend a separate scratch disk to the
operating system boot disk with a RAID 1 backup, an internal storage
disk for exported media, and an external storage disk set to
automatically backup the internal storage disk. The cost of a $4k
workstation means nothing if you lose a $50,000 job because of a
faulty hard disk. The kind of reputation you build from a catastrophe
like that can end your friend's business.

-D