TmpGEnc AVI Files Seem WAY Too Large

M

Martin

I wish to import some MPEG-2 files into Windows Moviemaker. Because
the problems with Moviemaker and MPEG-2 are well-documented, I decided
to use TmpGEnc to convert to uncompressed AVI as an intermediate
format acceptable to Moviemaker.

The source files are from a JVC Everio camcorder. This device writes
MPEG-2 files and tags them with a .MOD extension, which TmpGEnc does
not accept, but I found that all I had to do was change the extension
to .MPG and that problem disappears.

My question is the file size - the output AVI files are, on average,
about 1.5 Gigabytes per minute. Does this seem reasonable, and in
line with what an uncompressed AVI file should be? I know that mini-
DV files, which are not heavily compressed, are about 12 or 13 GB per
hour, or about 200 MB per minute. If uncompressed AVI is really
supposed to be 1.5 GB per minute, this would imply that mini-DV is
already compressed about 7:1, which I find hard to believe.

thanks and regards,

Martin
 
R

Richard Crowley

"Martin" wrote ...
I wish to import some MPEG-2 files into Windows Moviemaker. Because
the problems with Moviemaker and MPEG-2 are well-documented, I decided
to use TmpGEnc to convert to uncompressed AVI as an intermediate
format acceptable to Moviemaker.

The source files are from a JVC Everio camcorder. This device writes
MPEG-2 files and tags them with a .MOD extension, which TmpGEnc does
not accept, but I found that all I had to do was change the extension
to .MPG and that problem disappears.

My question is the file size - the output AVI files are, on average,
about 1.5 Gigabytes per minute. Does this seem reasonable, and in
line with what an uncompressed AVI file should be? I know that mini-
DV files, which are not heavily compressed, are about 12 or 13 GB per
hour, or about 200 MB per minute. If uncompressed AVI is really
supposed to be 1.5 GB per minute, this would imply that mini-DV is
already compressed about 7:1, which I find hard to believe.

DV25 (including "mini-DV", Digital8, DV, DVcam and DVCpro25) runs
at 25Mbps = 13.7 GB/hour. DV25 is spatially compressed 5:1 in the
camera but has no temporal compression (like many/most forms
of MPEG do).

1 Second = 3.5 MB
1 Minute = 215 MB
4 Minutes, 40 Seconds = 1 GB
1 Hour = 13 GB

http://people.csail.mit.edu/tbuehler/video/dv.html

Your rate of 1.5GB/minute is *way more* than even uncompressed
standard-definition television (NTSC or PAL). Sounds like something
is seriously screwed up with your configuration.

Remember that "AVI" is a container file and tells you nothing about
what is inside (i.e. which codec and parameters were used to encode
the video). Saying that you are trying to read an AVI file is like saying
that you are having Tupperware for dinner.

So what exactly are the output settings you are using with TmpGEnc?
 
K

Ken Maltby

Richard Crowley said:
"Martin" wrote ...

DV25 (including "mini-DV", Digital8, DV, DVcam and DVCpro25) runs
at 25Mbps = 13.7 GB/hour. DV25 is spatially compressed 5:1 in the
camera but has no temporal compression (like many/most forms
of MPEG do).

1 Second = 3.5 MB
1 Minute = 215 MB
4 Minutes, 40 Seconds = 1 GB
1 Hour = 13 GB

http://people.csail.mit.edu/tbuehler/video/dv.html

Your rate of 1.5GB/minute is *way more* than even uncompressed
standard-definition television (NTSC or PAL). Sounds like something
is seriously screwed up with your configuration.

Remember that "AVI" is a container file and tells you nothing about
what is inside (i.e. which codec and parameters were used to encode
the video). Saying that you are trying to read an AVI file is like saying
that you are having Tupperware for dinner.

So what exactly are the output settings you are using with TmpGEnc?


It's much easier to just use a MPEG Editor to edit MPEG.
http://www.ulead.com/vs/sysreq.htm

If that is too pedestrian for you then try Avid's Liquid.

Luck
Ken
 
F

Frank

I wish to import some MPEG-2 files into Windows Moviemaker. Because
the problems with Moviemaker and MPEG-2 are well-documented, I decided
to use TmpGEnc to convert to uncompressed AVI as an intermediate
format acceptable to Moviemaker.
Okay.

The source files are from a JVC Everio camcorder. This device writes
MPEG-2 files and tags them with a .MOD extension, which TmpGEnc does
not accept, but I found that all I had to do was change the extension
to .MPG and that problem disappears.
Great.

My question is the file size - the output AVI files are, on average,
about 1.5 Gigabytes per minute. Does this seem reasonable, and in
line with what an uncompressed AVI file should be?

I can't answer that without knowing the technical characteristics
(frame rate, frame size, color depth, audio type, etc.) of the .avi
file that you've created.

I have uncompressed .avi files with frame sizes of 80 by 60, frame
rates of 10 frames per second, and no audio track. I can assure you
that these files do not consume 1.5 GB per minute of storage space.

Here's an example that's perhaps closer to what you're dealing with: I
often work with hour-long (give or take a few minutes) uncompressed
..avi files having the following characteristics - 640 by 480 frame
size with square pixels, interlaced at 59.94 fields per second, 8-bit
RGB 4:4:4 color depth, and with 2-channel (stereo), 16-bit, 48 kHz
non-compressed LPCM audio. These files run about 100 GB per hour in
size, give or take a few gigs. If I run one of these 100 GB files
through a lossless compression algorithm such as HuffYUV or Lagarith,
the file size drops to about 50 GB, representing a 50 percent
reduction in the storage requirement but with no loss of visual
quality. If you have a space problem, and who doesn't, you might want
to consider use of a lossless video codec.
I know that mini-
DV files, which are not heavily compressed, are about 12 or 13 GB per
hour, or about 200 MB per minute. If uncompressed AVI is really
supposed to be 1.5 GB per minute, this would imply that mini-DV is
already compressed about 7:1, which I find hard to believe.

NTSC DV is indeed compressed at a 5 to 1 ratio but that's after 4:1:1
chroma decimation has been applied, so some people refer to DV as
having a real compression (or data reduction) ratio that's actually
closer to 7 to 1. I've never taken the time to sit down with a
calculator to confirm or deny the 7:1 figure, but certainly if you
take the 4:1:1 color reduction into account, the overall compression
ratio is more than 5:1, especially compared to a 4:4:4 signal, even
one with only 8-bit color.
 
R

Richard Crowley

"Frank" wrote ...
NTSC DV is indeed compressed at a 5 to 1 ratio but that's after 4:1:1
chroma decimation has been applied,

And 4:2:0 chroma decimation for PAL
 
F

Frank

"Frank" wrote ...

And 4:2:0 chroma decimation for PAL


Correct, DV25 is 4:2:0 in PAL-land, which is specifically why I wrote
a qualified "NTSC DV" rather than just "DV" or "DV25".

4:1:1 color sampling and 4:2:0 color sampling both store the same
amount of color information, it's just sampled in a different manner.

Of course, we don't know where the OP is located and exactly what kind
of .avi file he created. Personally, I don't think that his "1.5 GB
per minute" figure is far off, however, but lacking further
information, I can't really say. Even if he's in NTSC-land, I don't
even know whether he's dealing with square pixel 640 by 480 footage or
non-square pixel 720 by 480 footage, for example.
 
K

Ken Maltby

Frank said:
Correct, DV25 is 4:2:0 in PAL-land, which is specifically why I wrote
a qualified "NTSC DV" rather than just "DV" or "DV25".

4:1:1 color sampling and 4:2:0 color sampling both store the same
amount of color information, it's just sampled in a different manner.

Of course, we don't know where the OP is located and exactly what kind
of .avi file he created. Personally, I don't think that his "1.5 GB
per minute" figure is far off, however, but lacking further
information, I can't really say. Even if he's in NTSC-land, I don't
even know whether he's dealing with square pixel 640 by 480 footage or
non-square pixel 720 by 480 footage, for example.

Since I doubt there is any thing MS MovieMaker can do that
Ulead Video Studio 11 can't, I don't see why anyone would
be advising converting from the editable .mpg/.mod format to
a DV-AVI format, for editing in MS MM2.
(With apologies to the Moviemaker NewsGroup.)

If it were a matter of needing to use free software, there are a
number of programs that can read MPEG2 MP/HL video and
output AVI (DV-AVI with the codec). I would start with
VirtualDubMod myself.

But if I had such a camera, it would make sense to spend some
on editing software that could make it much easier to produce
the video results I bought the camera for. Without going through
time and potentially quality robbing conversion steps. I doubt
anyone buys a JVC Everio, as a professional production camera.
(Nor, were that the case, that MS MovieMaker would be the
preferred editing package. Apologies, again.)

The advantages of such a camera are more to a quick product
cycle, and ease of production for personal/home use. Besides
"Home Movies"/DVDs it should lend itself to a number of training
and event video productions that require a very rapid development
of distribution video.

Luck;
Ken
 
F

Frank

Since I doubt there is any thing MS MovieMaker can do that
Ulead Video Studio 11 can't,

I'm sure that there isn't, at least not much, anyway.
I don't see why anyone would
be advising converting from the editable .mpg/.mod format to
a DV-AVI format, for editing in MS MM2.

The OP wrote, "Because the problems with Moviemaker and MPEG-2 are
well-documented, I decided to use TmpGEnc to convert to uncompressed
AVI as an intermediate format acceptable to Moviemaker."

Apparently the OP decided upon this course of action on his own. I
certainly don't see anyone here advising him to transcode from one
lossy compressed format to another lossy compressed format (MPEG-2 to
DV-AVI as you've stated). Of course this thread is cross-posted to
microsoft.public.windowsxp.moviemaker and rec.video.desktop and since
I'm only reading it in rec.video.desktop, someone made have made a
suggestion that I missed.

As far as taking the lossy compressed MPEG-2 original and saving it
out as an uncompressed .avi is concerned, that's a perfectly good
thing to do if you've got the disk space, and it's a lossless process,
the loss having occurred back in the camcorder when the images were
shot and lossy compressed by the camcorder into MPEG-2 format.

In fact, if he keeps the footage in uncompressed format throughout the
editing process and saves the final result of his editing as an
uncompressed file, he's then got an uncompressed master file that can
be used to encode to any of a zillion different formats/codecs --
everything from RealVideo to Windows Media Video to an H.264/AVC file
for his iPod or PSP. Let the additional loss occur then.

Of course, if he's doing cuts-only editing with no transitions, fades,
wipes, color correction, titles, etc., then it makes sense to edit in
the native format, MPEG-2 in this case.
 
S

sgordon

: Your rate of 1.5GB/minute is *way more* than even uncompressed
: standard-definition television (NTSC or PAL). Sounds like something
: is seriously screwed up with your configuration.

Actually, rates of 1.5G/minute ARE common for uncompressed .avi files
if you don't use a codec. For example, if you take a DV file and load
it into Virtualdub, but don't specify a codec, that is exactly what you get.

Sometimes I find myself doing this temporarily if I'm mixing Virtualdub
and Vegas. Vegas has its own codec that can't be loaded into Virtualdub,
so at times I'll just go ahead and load a Vegas-produced .avi file into
virtualdub and after running a filter I get humongous-sized files. Of
course, I then load them back into Vegas and let it use its codec to
bring it back down to normal size. (I know that I could use HuffyYUV
codec in both, but it's only temporary and so far it hasn't been an issue
so I haven't gotten around to it.)

Sounds like for some reason your setup isn't utilizing a codec.
 
K

Ken Maltby

Frank said:
TmpGEnc AVI Files Seem WAY Too Large>,

Of course, if he's doing cuts-only editing with no transitions, fades,
wipes, color correction, titles, etc., then it makes sense to edit in
the native format, MPEG-2 in this case.

You should know by now that an MPEG editor like the
Ulead VideoStudio 11 I mentioned can do all those things
and many more, in native MPEG editing. Every thing
except the color correction will even be done with smart
rendering, a very fast process that only processes the
portion of the video impacted by the effect being applied.

But you should know all this, you are familiar with a number
of editors that can natively edit MPEG many like Avid Liquid
and other more expensive programs are used professionally,
to edit HDV (MPEG 2).

Some of the Womble products have been applying titles, fades,
wipes, and transitions to DVD compliant MPEG 2 for a very
long time now, several years at least.

Don't tell me you have bought into Crowley's MPEG phobia?

There is no need to put the perfectly editable MPEG2 through
several unnecessary time consuming conversions. There are
plenty of encoders that will create the file types you mention,
from MPEG2 input. That is certainly the case for H.264.
It would be odd, wouldn't it, if all the HDV (MPEG2) video,
couldn't be converted/encoded to any other format. You know
better.

Luck;
Ken
 
F

Frank

You should know by now that an MPEG editor like the
Ulead VideoStudio 11 I mentioned can do all those things
and many more, in native MPEG editing. Every thing
except the color correction will even be done with smart
rendering, a very fast process that only processes the
portion of the video impacted by the effect being applied.

But you should know all this, you are familiar with a number
of editors that can natively edit MPEG many like Avid Liquid
and other more expensive programs are used professionally,
to edit HDV (MPEG 2).

Some of the Womble products have been applying titles, fades,
wipes, and transitions to DVD compliant MPEG 2 for a very
long time now, several years at least.

Ken, I never suggested that your idea of editing the OP's MPEG-2 file
in an appropriate MPEG-2 editor was a bad idea. I was merely trying to
address the OP's original questions as to the storage requirements of
uncompressed .avi files and DV compression ratios. Using VideoStudio
Pro, Womble, etc. is a fine idea.
Don't tell me you have bought into Crowley's MPEG phobia?

LOL. I'm not even sure if I ever convinced him that a Redbook audio CD
contains no files or file systems. :)
There is no need to put the perfectly editable MPEG2 through
several unnecessary time consuming conversions. There are
plenty of encoders that will create the file types you mention,
from MPEG2 input. That is certainly the case for H.264.
It would be odd, wouldn't it, if all the HDV (MPEG2) video,
couldn't be converted/encoded to any other format. You know
better.

Again, I didn't comment upon your MPEG editor suggestion. I was
addressing the OP's original two questions. Your MPEG editor
suggestion was an excellent idea, although the OP seemed to want to
use Windows Mover Maker.

It isn't luck that I need, it's sleep. It's almost 3 AM here. I'm
watching the Science Channel. It's "space week".
 
R

Richard Crowley

...
You guys should form a comedy act. :)
Of course I have no "phobia" for MPEG.
I use it every time I master or dub a DVD video disc.
And even when I convert a file to put on my iPod.
LOL. I'm not even sure if I ever convinced him that a Redbook audio CD
contains no files or file systems. :)

I guess it depends on how you define "file system"
It certainly contains the basic elements sufficient for the
definition. (Else it wouldn't be practical.)
 
F

Frank

...

You guys should form a comedy act. :)

I can get you free tickets if you want to come see us perform, but to
get in you must know the secret password (hint: it's a four-letter
abbreviation that begins with the letter "M" and ends with the letter
"G"). :)
Of course I have no "phobia" for MPEG.

I never said that you did. That's Ken's part of the act.

But I certainly do recall some comments that you once made about how
no one doing serious work could possibly be using a camcorder that
utilizes MPEG as its acquisition format, a statement which is very,
very untrue. I also recall some more recently made comments, somewhat
positive in nature, about your impressions after viewing Sony's XDCAM
HD demo (DVD) disc. Assuming that they didn't fake the footage, and
even if it was mostly or even entirely shot at the 35 Mbps rate, it's
also long-GOP MPEG-2. Of course, it had to be downconverted for use on
a DVD, so we never got to see it in all of its high def glory.
I use it every time I master or dub a DVD video disc.

Don't we all.
And even when I convert a file to put on my iPod.

MPEG-4 in that case, I assume.
I guess it depends on how you define "file system"
It certainly contains the basic elements sufficient for the
definition. (Else it wouldn't be practical.)

As per the generally accepted definition of the term, there is no file
system (and no files) on a Redbook audio CD. Saying that there is a
file system (and files) on a Redbook audio CD is almost like saying
that there are files (and a file system) on a 12-inch vinyl LP disc.
Write to Philips (or Sony) and see what they say, or check various Web
references.

Getting back to Ken's point about using an MPEG-2 editor such as one
of the Womble products, I really don't know why he mentioned that to
me since I *did not* comment upon his recommendation to the original
poster, who hasn't made a reappearance in this thread, by the way, so
we may never know the actual characteristics of his .avi file (he
should run GSpot), but what I wanted to say was that I almost always
edit uncompressed and therefore care very little about how the footage
was originally acquired.

I like uncompressed footage for editing purposes because I'm lazy and
that way I don't have to concern myself with the negative impact of
compression artifacts caused by any possible recompression taking
place during the editing/finishing process. Also, in a sense,
uncompressed footage is easier to edit in that the processor load is
lighter due to the fact that it doesn't have to decompress frames on
the fly (especially taxing with long-GOP MPEG-4 Part 10 H.264 AVC),
although I'll certainly grant that a fast and large hard disk I/O
subsystem is mandatory, especially for editing HD.

I take this same uncompressed approach to almost all of the media that
I deal with, whether audio, video, or still images. I generally
decompress any lossy compressed format that I receive to an
uncompressed form and edit the non-compressed file and don't create a
lossy compressed version unless and until I absolutely have to. In the
case of audio - say someone sends me an .mp3 file to work on - I
create an LPCM .wav file from it and work entirely on that. If I'm
given a .jpg image file to work with, I'll immediately open it and
save it out as a .bmp or .tif and work on that, etc.

With regard to the HDV format and its use of long-GOP MPEG-2 video
compression and MPEG-1 Layer II audio compression, I was initially
excited three years ago when I first read about the format (under $10K
HD for the masses), but was *severely* disappointed when I saw the
technical specs - the MPEG-2 video and the MPEG-1 Layer II audio -
because I had always considered those formats to be distribution
formats and not acquisition formats. Just because I created, and
continue to maintain, a (rather lengthy) Web page devoted the subject
of HDV doesn't mean that I am totally, completely, and absolutely in
love with the format.

Sony has at least two new HD camcorders coming out within the next
year, one of which will be a full-sized, over-the-shoulder HDV-format
product. This camcorder, which hasn't yet been assigned a model
number, will be a tri-mode DV/DVCAM/HDV product, have three CMOS
sensors (probably 1/3-inch but maybe 1/4-inch), will support the use
of standard (large-size) cassettes (276 minutes of recording time on a
single tape), and will probably have an HD-SDI output jack. Pricing
hasn't yet been disclosed, but it's expected that it will be
positioned as a DSR-250/DSR-250P replacement. I haven't touched one,
but I've seen photos and it appears to be designed as a professional
product in terms of control layout. It will probably turn out to be a
nice companion to Sony's pro-grade HVR-1500 HDV VCR.

The other HD camcorder that Sony will be coming out with in the next
year (supposedly this Fall, actually) is the so-called XDCAM EX
product. The under $8000 price tag looks interesting, but it's still
4:2:0 because it doesn't support Sony's new 4:2:2 50 Mbps codec that
will be included on some future XDCAM HD products (which will also
support the use of Sony's new dual layer Professional Disc). Instead,
it will support the usual XDCAM HD 18 Mbps VBR, 25 Mbps CBR (1080i
HDV-like), and 35 Mbps VBR 4:2:0 rates. If it did support the new 50
Mbps 4:2:2 codec, I'd almost buy one myself just for the heck of it.
Unlike the new HDV product mentioned above, it will not be a
full-sized, over-the-shoulder camcorder but instead more of a Handycam
type of product. It will use three 1/2-inch CCDs (not CMOS, or so I'm
told) sensors, have a fixed (non-interchangeable) manual Fujinon (not
Zeiss) 14x zoom lens, offer variable 1 to 60 fps rates for effects,
and will record to the new Sony/SanDisk SxS ExpressCard/34 flash
memory cards (two slots provided). It's expected to support
1080i59.94, 1080i50, 1080p29.97, 1080p25, 1080p23.976, 720p59.94, and
720p50 but not 1080p59.94, 1080p50, 720p29.97, 720p25, or 720p23.976.

I don't want to talk about AVCHD.

Don't know why I wrote most of the above, actually. Some sort of
stream of conscious thing I guess. Must be over-tired.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top