How to mix HDV and AVCHD on the Same Final Cut Time line

Originally I planned to use Roxio’s Toast 10 to import AVCHD footage into HDV format video and then edit that, together with other actual HDV video, using Final Cut Pro version 5 (which does not support AVCHD).

However, the Toast 10 feature to import AVCHD and convert to other formats does not work – in fact, per Roxio support, the AVCHD import feature is broken and produces video that is not synchronized with the audio.

I have, however, found a work around that works well for me – and does not use Toast.

I am using iMovie to import the original AVCHD – this automatically transcodes to QuickTime movie files containing Apple Intermediate Code (AIC) encoding.

Then, run Final Cut Express HD (version 3.5.1). FCE HD supports HDV, it turns out, by converting HDV to AIC – so the clips are stored in AIC format. (Final Cut Pro, on the other hand, works with HDV encoding directly and so imports HDV as HDV.)

Next, import the video clips created by iMovie (in the Movies, Events folders – or other location if you chose a different location during import) into Final Cut Express HD. Once in your bin of clips, you can drag these clips directly to the FCE time line. Since the time line is set for AIC encoding, there is no need to render from one format to the other.

I am fortunate that I kept the older Final Cut Express HD on my computer after upgrading to Final Cut Pro. Combined with the new version of iMovie – which automatically handles AVCHD conversion to AIC – this works very slick.

I’m going to see about returning Toast 10 and getting a refund. Toast 10 is very buggy and some of the features claimed for the product on the Roxio web site do not actually work.

Separately, it is probably possible to import HDV and transcode to AIC in Final Cut Pro version 5. I haven’t tried that yet but I think of the option settings likely required to make that work. Then FCP could be used to do the same editing, all in AIC. (You can set the timeline codec to AIC so that you do not have to constantly render the video.)

Working with AVCHD on the Mac

I do nearly all of my video editing now on a Mac so this post is about processing AVCHD files on the Mac.

Easiest Import

The easiest way to deal with AVCHD is to let iMovie (latest version) import and transcode to the Apple Intermediate Codec (AIC). If you want, you can then edit in iMovie. If you wish to edit in older versions of Final Cut Express or Final Cut Pro, you can import the Quicktime .mov files created by iMovie. Set your timeline sequence encoding to the Apple Intermediate Codec. You can now edit copy your clips direct to the time line, edit as much as you want.

Converting AVCHD to HDV Format

If you must edit both original AVCHD video and HDV video into a single time line and do not want to spend lots of time rendering video clips from AIC into your HDV timeline, then it may be useful to convert AVCHD direct to HDV. If you do not need to merge with HDV footage, then you will not want to do this at all!

Toast 10 from Roxio is the only product that has successfully done the transcoding from AVCHD to HDV (for me). Toast can import your AVCHD files directly from the camera and then re-encode into HDV file format. You do lose some quality (but not very much that most people would actually notice) by re-encoding into HDV.

UPDATE 2009: The Toast 10 product has been riddled with bugs (defects). The company  came out with a 10.0.2 fix release five months later but much is still broken. Worse, the AVCHD conversion feature in Toast 10 DOES NOT WORK. Their decoder gets the video and audio out of synchronization. It does not matter whether you convert from AVCHD to AIC or to HDV. The synchronization will be off. Roxio knows about the problem but after many months has not yet produced a fix. I recommend that you do not purchase Roxio Toast 10 at this time. (Its now 2012 as I revise this text – presumably this has been fixed but I have not gone back to check.)

24p

If you have a consumer camera with 24p capability, you probably cannot directly edit the 24p video without first doing a conversion. But before you consider shooting in 24p, ask yourself if you’d be better off shooting 30p progressive? If you have sort of last year’s Canon camera (HV20, HG10, HG20), you can, in fact, shoot 30p progressive using a simple trick:  Use the Program mode and the Tv selection to set your shutter speed. Set the shutter to 1/30th of second. Your camera now records in 30p mode! This is because the camera actually contains a progressive imaging system-with a 1/30th second shutter speed, both “fields” of the 60i video are recorded without any field shifts. You get perfect 30p, as long as the 1/30th of a second shutter speed is acceptable. (Most 24p is shot at 1/48th or 1/24th of a second. Many people like the blurred motion effect of the slow shutter speeds. Your choice). This trick actually works on a wide variety of older cameras too. If you have a newer camera with true 30p, you may wish to use 30p for all your shooting since it can be directly processed for online or Apple TV/XBox360/PS3 viewing without having to de-interlace.

If you do use the 30p trick, just import your video as 60i and edit as usual. That’s all there is too it.

But if you choose to use 24p, then you must do a “reverse telecine” (or sometimes called “de-telecine” or “inverse telecine”) step to convert the recorded 24p to a true 24p that can be edited in Final Cut Pro. (Final Cut Express and iMovie do not edit 24p video).

VoltaicHD ($35) claims to do the conversion but the demo version did not do so successfully for me. However, if it works, you can use VoltaicHD to import and do telecine removal in one step, producing the AIC files for editing. If it worked, this would be my preferred solution.

HandBrake (see next section) also claims to do reverse telecine conversion but it failed to do so properly on my HG10 video stream. However, its output format is MP4 which is not ideal for video editing.

JES De-Interlacer does do reverse telecine and does it properly and very well (and also does excellent de-interlacing as well as several other features). And its FREE! To use, first import the 24p AVCHD files in iMovie. Then, find the converted files (.mov) in your Movies Events folder. Run JES-Deinterlacer, select all of your input files, and the options page, select the “inverse telecine” option. On the output page, specify a folder to store the converted files – and set your output file format to AIC (Photo JPEG is also very good). Then let it run to convert the input files, producing a new set of output files.

You can then import the output files to Final Cut and begin editing.

You can also see why editing 24p is viewed as somewhat difficult!

Update Fall 2012

Since this post was originally written, another popular solution has emerged called Neo Scene which converts videos to the Cineform codec (now owned by GoPro). Neo Scene converts your AVCHD into the Cineform codec – files do become much larger but editing becomes much quicker and smoother. The Cineform codec is better suited for editing than is AVCHD and many editing programs can play Cineform coded video in real time, smoothly.

I do not – yet – own this product but have used the trial version and have used the free GoPro Studio (which also uses Cineform but does not support so many cameras). Cineform really works.

Cineform can also handled 24p files embedded in a 60i video stream. Versions are available for both Windows and Mac, and based on my experiments on Windows – Cineform coded files work fine in Sony Vegas Movie Studio 11 and in Magix Movie Edit Pro latest versions.

Converting Camera Video Files Direct to MP4 Format

Plug in your camera and copy the files from the camera to your hard disk. (You can also use iMovie’s Archive feature to do this for you).

Then, run the HandBrake program. It can read AVCHD files and convert directly to MP4 files. HandBrake is quite fast and can be configured to re-size your images to 1280×720, as well as de-interlacing (if you did not shoot in progressive). HandBrake is probably the fastest (and FREE!) utility for converting your camera files direct to MP4 files for viewing XBox360’s, Apple TV etc.

Toast 10 can also do the AVCHD to MP4 conversion in one step.

Working with AVCHD video files

For those accustomed to working with miniDV standard definition video tapes or the HDV 1440×1080 high definition format, the switch to AVCHD may prove challenging from an editing perspective.

“miniDV” and HDV formatted video can be edited directly on most computers with the caveat that editing HDV can take 4 to 5 times longer than mini-DV simply because there are a lot more bits to manipulate for the high definition video. For editing HDV and AVCHD you definitely want at least a dual processor (or dual core) and a minimum of 1 GB RAM (2 or more recommended).

mini-DV records video and audio at a rate of about 25 Mbps. Each video frame is compressed and then stored on the tape, individually using MPEG1 compression.

Summary

  • miniDV image size is 720×480. Data rate is 25 mbps.
  • HDV image size is either 1280×720 (few cameras use this) or 1440×1080. To display an HDV image on a 1920×1080 display, the image is stretched horizontally by 33%. Data rate is 19.7 Mbps (1280×720) or 25 Mbps.
  • AVCHD image sizes are 1280×720, 1440×1080 or 1920×1080 – with a wide variety of data rates from as low as 5 Mbps up to 24 Mbps.

HDV

HDV records video and audio also at a rate of about 25 Mbps. To record a much more detailed image, HDV uses a  compression scheme based on MPEG2. Unlike miniDV, which stores each individual compressed frame, HDV stores a compressed full frame, and then follows that with 14 frames recording only the changes from the first frame.  Since the HDV codec only stores a full frame once every 15 frames, it can do a remarkable job of compressing a 5x larger image into the same space as standard definition recording. (There is also a 19.7 Mbps HDV version for the 1280×720 image size – however, there are few if any any consumer video cameras that record in 1280×720 mode on HDV so we will ignore that.)

The 15 frame sequence is known as a “group of pictures” or GOP. There is also a version with a 7 frame GOP.  While HDV works very well, a major disadvantage is that a single tape drop out will typically ruin up to a full 15 frames – or 1/2 second of video. In the older standard definition encoding, a single dropout killed one frame or 1/30th of a second and no one noticed. With HDV, everyone notices a tape dropout!

AVCHD

AVCHD uses a version of MPEG4 encoding (H.264 or MPEG4 Level 10) that is twice as efficient as MPEG2. That means a 15 Mbps AVCHD encoding is roughly equivalent to a 30 MBps HDV recording (if such a thing existed). However, AVCHD can support multiple image sizes up to the full 1920×1080 HD resolution – plus 1440×1080 and 1280×720.. A 1920×1080 image requires 33% more bits than does a 1440×1080 image.

Most AVCHD based cameras record to Flash memory, and some to hard disk memory. Tape dropouts are then a “thing of the past”. This is a major plus for AVCHD. (Note – you can buy an expensive add on that will record HDV direct from the camera to hard disk to avoid tape drop outs – but these are expensive!) The major negative of AVCHD is that editing is very time consuming or requires huge disk space. And while tape dropouts are gone, you can still get corrupted file problems on the storage media (it happens to everyone eventually).

Editing AVCHD

Editing standard definition video is straightforward since the original video contains each and every frame. Editing HDV and AVCHD is more complicated because the original video does not contain each frame. Further, AVCHD uses a complex encoding scheme that takes a lot more processor time to encode – and decode. Consequently, no matter how you slice it, AVCHD can take longer to edit.

At this time, there are very few editors available that can edit AVCHD files directly (Sony Vegas being a notable exception that can edit AVCHD). Thus, most solutions require that the AVCHD files be converted to an editable format.

On the Mac, the easiest solution is to connect your camera and let iMovie (or Final Cut Express 4 or Final Cut Pro 6) automatically import and convert the AVCHD format files to the Apple Intermediate Codec (AIC) or ProRes 422 codec format. However, converted files expand by 3 to 5 times larger than the original! Where as an hour of HDV or SD video might be less than 13 GB, an hour of AVCHD might be 49 GB! Fortunately, hard disks are now cheap.

So the disadvantage is disk space – but the advantage is very fast editing.  If you have an older version of Final Cut Express (e.g. HD or 3.5) or Final Cut Pro (e.g. 5), you can use iMovie as your importer and then directly import the resulting converted files in to Final Cut (or just edit in iMovie if you want). If you set your time line sequence to AIC codec in Final Cut, you can import and edit all the files directly without a “rendering” step.

On Windows, the most straight forward solution is to use Sony Vegas – or the newest version of Adobe Premiere Pro CS4 or Premiere Elements 7 (update: or Magix Movie Edit Pro or Pinnacle Systems video editor). The advantage to direct AVCHD editing is very fast access to the files – the disadvantage is that each transition (dissolve for example) or title may require that the AVCHD be decoded into individual frames at edit or render time. So the time issue just gets pushed to a different part of the editing workflow.

Other wise, you’ll need to buy an application like VoltaicHD[1], which will convert the AVCHD files to AVI files that can then be imported to other editors (but which may require a rendering step in the editor). The files will also become a lot larger after conversion.

Some AVCHD capture programs enable you to view thumbnail clips of the original AVCHD that is still on the camera – and then select to import only the clips you want to work with (iMovie, Roxio Toast 10). Many also provide a way to quickly copy the compressed AVCHD files and archive those files – either by writing them to hard disk or, for some, burning to a DVD data disc for long term storage.

UPDATE TO NEXT PARAGRAPH: ROXIO TOAST 10 IS DEFECTIVE AND IS UNABLE TO CORRECTLY PROCESS AVCHD FILES. IT WILL PRODUCE FILES WITH UNSYNCHRONIZED AUDIO AND VIDEO.

Another method that works on the Mac is to use Roxio’s Toast 10 software. It can read AVCHD files from your camera and convert them to any format – including HDV or NTSC DV for subsequent editing. Obviously, if you decode a compressed AVCHD file and then re-encode into another compressed format like HDV, you will lose a tiny bit of image quality. Depending on the original source material, most people won’t notice.

Finally, there is the issue of dealing with 24p – or 24 frames per second mode. Depending on the camera and the editing software used, you may have to perform an “inverse telecine” operation on the original 24p video prior to editing. See footnote 2.

Update Fall 2012

Since this post was originally written, another popular solution has emerged called Neo Scene which converts videos to the Cineform codec (now owned by GoPro). Neo Scene converts your AVCHD into the Cineform codec – files do become much larger but editing becomes much quicker and smoother. The Cineform codec is better suited for editing than is AVCHD and many editing programs can play Cineform coded video in real time, smoothly.

I do not – yet – own this product but have used the trial version and have used the free GoPro Studio (which also uses Cineform but does not support so many cameras). Cineform really works.

Cineform can also handled 24p files embedded in a 60i video stream. Versions are available for both Windows and Mac, and based on my experiments on Windows – Cineform coded files work fine in Sony Vegas Movie Studio 11 and in Magix Movie Edit Pro latest versions.

Working with 1920×1080 AVCHD

On the Mac, if you are using iMovie as your editor, you need to be aware that the AIC codec only stores 1440×1080 images. So when you import your 1920×1080 images, they are re sized to 1440×1080 and then stored in the AIC format.

Does it matter?

Reading online forums, I see many people are focused on their 1920×1080 pixel resolution of their new Flash-based camera – but neglect that many consumer lenses cannot deliver the full resolution anyway. And after transcoding and editing, they then generally output their final production to 1280×720 video for online upload, or display via AppleTV, Sony PS3 or XBox 360. Or they burn a standard definnition DVD at 720×480.

What that means – 1920×1080 may not be as important as you think. Yes, 1920×1080 recorded on your camera, and then hooked up to an HDTV via an HDMI cable will look super! But only if you play back from the camera.

If you edit, transcode and ship as 1280×720 or standard DVD, no one will be able to tell the difference if you recorded your original in 1920×1080 or 1440×1080 or 1280×720.

What is more important to image quality is the person behind the camera making good decisions about what to photograph and how to photograph – and using a tripod of monopod wherever possible for steady images. Avoid excessive fast pans and fast zooms – which can make your highly compressed media turn to mush trying to deal with so much rapid motion.

—————–

Notes

[1] There are a bunch of Windows-based programs that claim to convert AVCHD to other formats. Oddly, the programs are nearly identical except for minor changes to their user interface. Most seemed to be based on the freely available ffmpeg program (but which does not support AVCHD) with some AVCHD decoder added on. The programs are so similar, that I suspect they are all distributed by the same organization under different names (but all priced at $35) as some sort of market strategy. Who knows? Anyway, VoltaicHD is its own program and not part of the mass distribution of similar programs. I’ve run a demo of VoltaicHD and it worked fine.

[2] Working with 24p. Some of the newer cameras offer 24 frame per second shooting mode. Some, like Canon’s HG10, HG20, HV20/HV30 stuff the 24 frames into a 60-interlaced fields sequence using a technique known as 2:3 pulldown (which I will not try to explain here). To edit this type of 24p, you must first pre-process the video (which otherwise resembles normal 60i video) to remove the pulldown frames prior to editing. The VoltaicHD program has an option to do this automatically during the transcoding from AVCHD step – which will save you time in the longer run. Otherwise, for example, I have imported using iMovie, but then run the imported videos through the JES Deinterlacer using it’s pulldown removal option (also known as “inverse telecine”), which adds another step and more disk space to the issue of shooting in 24p!

HD Camera deal

If you do not have an HD-capable consumer video camera and are interested in getting one, OneCall.com (based in Spokane) has quite a deal going at the moment.

The older Canon HG10 40 GB hard disk based high end consumer HD video camera is on sale as a Canon factory reconditioned unit, with limited warranty, for $419.  Over the weekend it was $399 and I bought one then.

The HG10 records up to just under 6 hours in its highest quality HD mode. It has manual overrides for most everything, which puts in the higher end consumer video category. It has an external mic input too.  Its a Canon so uses all the same Canon icons on its menus – same as the still cameras.

Link: http://www.onecall.com/ProductDetails.aspx?id=98107

This is a complete and fully usable camera out the door whereas the nearly equivalent Flash cameras sell for $600 to $1200 dollars – and then you need to buy 32 GB Flash cards at $128 a piece. And probably at least a couple. Plus extra batteries (regarding the HG10, it uses the same batteries as the HV20/30/40 and I already had some of those).

The feature set and image quality are comparable to the widely successful Canon HV20 (since replaced by the HV30 which added true 30p capability and then replaced by the HV40 that updated the 24p mode to true 24p mode). With one exception: some scenarios can produce visible compression artifacts in AVCHD that are not apparent in HDV. Why?  The problem is that Canon implemented an efficiency measure in the HG10’s version of AVCHD compression that, where it sees fairly broad expanses lacking in detail, it reduces the compression effort by processing 8×8 pixel matrices rather than the usual 4×4 pixel matrices. Much of the time, this means that it can assign more bits to the detailed parts of the image – but it has the odd effect, especially on a graduated color such as the blue sky from the horizon upwards, where it can produce compression artifacts on the sky while getting on the really detailed stuff nailed. Weird! For my uses, this is not a great problem. But it is something that anyone using AVCHD systems probably needs to be aware of.

The imager is a progressive 1920×1080 (4:2:2 color space), downconverted to 1440×1080 4:2:0 color space AVCHD at 15 mbps. The camera records in 1080/60i and 1080/24pf mode. The “pf”reference to “progressive frames” and is identical to 24p. You can record in a fake 30p mode by using a 1/30th of a second shutter speed. This works because the progressive imager puts the correct fields into the /60i video stream such that each field is lined up correctly as if it were true 30p.

Quick run down on HD:

  • Most HD cameras for the consumer space are either tape-based (e.g. Canon HV40) and record in 1440×1080 HDV (MPEG2) or Flash memory based.
  • TAPE BASED: This is the easiest format to edit on the computer, but if you record one hour of video, it takes one hour to ingest into the computer before editing.  Consumer quality tapes result in tape dropouts – which are a killer in the HDV format as one drop out can take out 1/2 second of video. I only buy professional quality tapes and they are not cheap ($5 to $8 each depending on quantity) but they can be reused.
  • FLASH BASED: Usually records in 1440×1080 or 1920×1080 AVCHD (H.264/MPEG4). In spite of more pixels, AVCHD is up to twice as efficient as HDV. Thus, in theory, a 15 mbps AVCHD stream would be better than HDV’s 25 Mbps data stream. In theory.
  • The Canon HG10 is a generation behind and it records 1440×1080 anamorphic HD in AVCHD – which is exactly the same as the tape based HDV format.
  • Does 1920×1080 actually matter?  After reading numerous online discussion groups, virtually everyone who raves about their super sharp 1920×1080 images ends up editing and encoding for online distribution as 1280×720 or for playing through their Apple TV/Sony PS3/Xbox 360 to their HDTV!!! Or re-encodes to standard DVD format at 720×480. In such circumstances, no one will notice the difference between 1920×1080 and 1440×1080 in the final result.
  • Yes, its true – the BBC and Discovery Channel now require most of their shows to be shot at 1920×1080 and do not usually use 1440×1080 video. So for all those people who plan on shooting BBC documentaries with their $700 flash camera – well, you can worry about that 🙂
  • The newest Flash cameras such as the Canon HF200 uses a tiny 1/4 sq inch sensor with something like six mega pixels on it. Versus a 1/2.7 sq inch imager with 3 megapixels for the HG10. What do you think happens to low light performance of a 1/4 sq inch 6 megapixel sensor versus a larger image sensor? Well, its terrible and becomes very noisy as the video gain (amplification) is turned up to high.
  • But isn’t tape good for archiving? All of us used to think we wanted to have our video archive on tape. But by late last year, it has become cheaper to store an equivalent amount of video on hard disk rather than tape!
  • Editing – on Windows, the only package really good for editing AVCHD is Sony’s Vegas, in either the consumer or professional editions. It will edit AVCHD directly. CAUTION: No matter what editing software use and what type of computer you use, you realistically need at least a dual processor. HD has a lot of bits to process! (Update: Adobe Premier Pro CS4 and Adobe Premier Pro Elements 7 will also edit AVCHD directly.)
  • Mac OS X – iMovie 08 and 09 work fine (09 recommended). iMovie reads the AVCHD and converts it to an intermediate form called Apple Intermediate Codec or AIC which you can then edit at normal speeds. The intermediate file size is usually about 3x larger than the original AVCHD. But you can archive the original AVCHD and then delete the AIC files after you are done editing, if you want.
  • Another alternative on either OS is to use the TOAST 9 or 10 (Update: Toast 10 is defective and does not work) software which can be used to read the AVCHD files on the camera and re-encode in to your preferred editing format, including HDV for compatability with HDV editors or mixing with other HDV footage. (Yes, you lose a tiny bit of quality in the re-encoding as HDV is a lossy encoder but most people won’t notice unless you are shooting for the BBC…)
  • All editing tools let you view thumbnails of the videos on the camera, and then import only the ones you want. You may optionally import and archive the compressed AVCHD to a hard disk for longer term storage.

Drawbacks

For me, the one major limitation of the HG10 is that there is no way to view or set the microphone audio level. I am used to manually setting the audio level – and I prefer to do this to keep the sound level at constant setting to property record the dynamic range of the subject. Unlike the HV20/30/40, which are so similar to the HG10, there is no way to adjust the mic input level – instead, it uses automatic gain adjustment. It is possible to slightly accommodate this by using an external BeachTek audio mixer (or similar) – but the in camera ALC may still interfere with your desired audio level setting. You can use an external mic, such as the R0DE short shotgun with mini plug – but you’ll probably have to rely on the automatic limit control buil in to the camera.

For some, and probably for the wrong reasons, the 1440×1080 format is less resolution than 1920×1080 and this may be an issue. But most people can’t tell the difference and especially once it has been edited and encoded for distribution at 1280×720. Or heavily transcoded for 1280×720 online!

The camera’s 24pf mode requires removal of the 2:3 pulldown sequence. On the Mac, this is done using the free JES Deinterlacer. I believe it can also be done on both Windows and the Mac using the free MPEG Streamclip program.

The camera lacks a 30p mode but this can be simulated in many situations by shooting at 1/30th of a second – provided that the subjects are slow moving – definitely not for sports unless you want a lot of blur! This trick does work and works very well – again provided your are not shooting sports or other fast action!

I am told that the hard disk is limited to a total of 1,000 separate clips maximum, regardless of disk space available. Because of how I use the camera, I do not see how I would ever encounter that. But your mileage may vary on that point.

If you really want to shoot for the BBC, then go buy yourself a Sony EX1, 2 or 3! Actually, there are some other solutions including a JVC camera that records 1920×1080 into Quicktime files using the XDCAM 35 mbps format.  Or the Panasonic HMC-150. But for the money and for most of us back in the real world, the generation behind HG10 is quite a deal.

The same factory reconditioned unit sells in the Canon online store for $549.