How to disable video gain on the Canon PowerShot SX1 camera

The Powershot SX1 is an amazing camera. I bought mine factory re-furbished, at a tremendous discount.

But it has a two frustrating gotchas. Both are its poor low light capability due to its small image sensor and lack of good signal processing to clean up the noise.

  • For stills, avoid shooting images above ISO 200 unless you have to; and then, only go to ISO 400. Anything above that is unusable. You can use third party software like Noise Ninja or NeatImage to really clean up camera noise, but would be nice to avoid that step. Never use the “Auto” mode in low light – it tries to turn dark into daylight. Instead, use P, Tv, Av or M modes. Set your ISO manually and/or set your own exposure settings.
  • Similarly, its low light video capability is positively awful and there is no way to override the camera’s automatic video exposure settings.

Or so we thought. Today I discovered a trick to affect the video gain setting.

For those of you who are still photographers, the ISO setting of your digital camera is equivalent to a video camera’s video gain. The higher the ISO number, the more amplification is added to the signal to try and pull an image out of darker scenes. Amplification occurs to both the useful signal – and the noise on the image sensor – such that increasing the amplification results in worse noise.

Professional video cameras and higher end consumer video cameras have a video gain setting to select, typically, video gain of 0 db to +18 db (some go as high as +36 db gain). The greater the number, the more gain or amplification you are adding to the signal and the noisier the image. But this means on real video cameras, you can disable gain (and the ugly noise) if you don’t want to have that.

Now for the trick. I have CHDK installed on my SX1 so the screen might show something different if you do not have CHDK running.

I pointed the camera out the window to where the sun was shining. I pressed the video record button. I then pressed the photo exposure button half way until the white focusing rectangle turns green (or yellow if you pressed the +/- exposure compensation button first). Continue to hold the exposure button half way down. This shows the ISO setting the camera has selected – for video.

Pan the camera into the interior of the room while continuing to hold the exposure button half way depressed. The room will remain dark and the ISO setting stays at 160 (or whatever it was for you).

Now, with the video still running, let go of the exposure button. The auto exposure feature kicks in and the ISO jumps to 400 and the room brightens up.

Therefore, the trick is to start your video clip at a bright area or light. Press video record. Then half depress the photo exposure button until the green rectangle appears. This locks the exposure and ISO setting (gain). Now, pan the camera into the area you want to video without so much awful noise caused by jumping to a high ISO (gain) setting.

Pretty cool, huh?

For interior shots, the SX1’s auto-only exposure mode tries to go to ISO 400 for everything. The total ISO video range is from ISO 80 up to a displayed 400. The ISO 400 setting is what shows in the display but I think its pushing out to about ISO 3200 when it goes full open. It is way too noisy and way to bright for ISO 400.

I found the best images occur with the ISO set to anything below 250. ISO 80 to ISO 125 is best, but its okay to 250. The next setting level is 320 and that starts getting pretty noisy at that point.

I ran across this by accident.  But it is a modified version of the exposure lock trick discovered on Canon’s HV20/HV30/HV40 camcorders. Same idea – point the camera at a brighter light and then invoke the exposure lock feature on the HV camera and you are good to go with video gain completely disabled. (On those cameras, you can also use the “Spotlight” mode, which disables video gain.)

Another trick that helps a little, and only a little, is the exposure compensation button in video mode. Start you video. Press the +/- button and it displays an exposure compensation setting bar (left to right) at lower left of viewer. Use the control wheel to reduce the exposure or increase the exposure. This affects the ISO but only a little.

Why didn’t Canon enable manual override of ISO, aperture and shutter speed? Certainly would have been possible. And might even be possible with CHDK if we can figure out what memory ports to poke with what values.

“Aliasing” and “Moire”

The new “HDSLRs” do a fine job of shooting very nice HD video. But almost all are plagued by several problems that can ruin a good shot.

These problems are “rolling shutter” (previously written about on this blog), and “aliasing” and “moire”. Aliasing tends to introduce fake detail into an image, fooling people in to thinking its a sharper image than it really is – because its fake. Moire produces bizarre color patterns wherever there are narrowly spaced, using horizontal lines in the image.

This Barry Green article is the best explanation I’ve seen for explaining and illustrating the aliasing and moire problems with DSLRs shooting video:  DVXuser.com – Articles.

A great visual example of a moire pattern showing up in a real life situation is this video clip – about 30 seconds long. Just look at the roof of the hotel and watch all the false color shimmering going on. This example was shot on a Canon DSLR.

Here’s another example – look at the sofa cushions:

canon eos 60d video “moiré” problem at 1280×720 50 from emre karabakan on Vimeo.

Using the Roku Internet video box with a USB Drive

I bought a Roku box last month and its been quite a bit of fun. When we watch TV, we now watch Vimeo or Netflix. There is a lot of great “user created” content – in fact, I have not watched satellite TV in six weeks now and plan to terminate our subscription this coming weekend.

The new Roku box also supports an external USB drive – which could be either Flash or hard disk based.

Streaming my own videos from the drive has mostly worked fine. Except for some that pause frequently and say “loading, please wait”. Looking around the net, I found a number of mostly random explanations.

The simple reason is that the data rate of the videos I created (from my own video productions) was too high. I had encoded 1080/30p videos at 16 Mbps (same as BluRay discs).

I registered as a software developer (in real life I am a professional software engineer) with Roku and have downloaded the developer documentation.

According to the documentation, Roku supports .mp4 files, mov and .m4v files having h.264 video compression, and WMV9 SD and VC-1 HD video files, using constrained variable bit rate (VBR) encoding. (I have not succeeded in getting the .wmv files or .mov files to work, however.)

The frame rate must be either 23.976 or 29.97 fps and this is probably the key part – the average USB disk video bit rate is limited to 8 Mbps depending on the codec used and the peak video bit rate is limited to 1.5 x the average bit rate. (For streaming videos coming over the Internet, the data rate is limited to an average of 1.6 to 4 Mbps depending on the coding. For H.264 HD, its limited to an average of 3.2 Mbps.)

Depending on your MP4 encoding tools, you might have difficulty specifying precisely the options you want. The key goals, it appears, are to try and select a constrained VBR, set to less than 8 Mbps, at 23.976 (a.k.a. 24 fps) or 29.97 (a.k.a. 30 fps).

I am doing some experiments with encoding at different data rates to see what happens and I will probably post an update later.

One thing that is clear – encoding a 1920x1080p video is sort of a waste of time since you’ll have to encode it at less than 8 Mbps. At that level of compression, a 1280x720p at 8 Mbps will actually look much better. (Well, unless your video is mostly static images with little movement.)

Consequently, while the Roku box really does support 1080p, you are unlikely to see video clips that are actually any better than 720p. In fact, most people cannot tell the difference between 720p and 1080p anyway, especially if the screen is less than 55 inches.

Update: On Mac OS X, I used MPEG Streamclip to create some 1280×720 video files for playing off a USB drive with the Roku. At about 7.5 Mbps, the video almost plays without buffering. I probably need to pick a slightly lower data rate – may be 7.2 Mbps for the video clip.

I had much better success playing some 8, and even 12 and 16 Mbps encoded videos produced by the Magix Movie Edit Pro 17 program on Windows 7.

I am guessing that it is not strictly the data rate but the “complexity” of the h.264 encoding used as well as whatever the unspecified “peak” data rate might be. MPEG Streamclip does not provide many options to control the h.264 encoding while Magix provides overrides of just about everything. Again, I’m guessing here but I suspect that MPEG Streamclip might be using one of the higher complexity profiles, or setting the “peak” rate to perhaps 2 x the selected data rate, and the Roku just can’t keep up.

I do think we’ve identified why some of us have received the “loading” message periodically while trying to play our own videos off a USB drive using the Roku box. The solution is to encode at a data rate probably between 7 and 8 Mbps using a variable bit rate h.264 encoder and setting the peak data rate, if you can set it, to something less than 12 Mbps.

2011 Camera Predictions by Thom Hogan

2011 Predictions by Thom Hogan.

Having just returned from the 2011 CES, I find myself in general agreement with his predictions. I also see he has a Spokane, Wa connection; didn’t realize that.

Worth reading what Thom has to say – whether you agree or disagree. My sense from CES is that Sony and Panasonic are mostly heading in the right direction in the future, except for Panasonic closing off the hacking door on the GH-1. Very, very dumb move. I wish the camera makers would have their DSLR staff talk more with their video staff. They make great new DSLRs but leave out things like on screen zebras, audio level controls, or auto focus lenses – or if they have them, the focusing mechanism is noisy, which doesn’t work in audio/video!

And no one seems to have a DSLR with video that also includes a simple headphone audio output – monitoring your audio is usually pretty critical!

Canon, Nikon and the traditional camera makers seem bent on ever more pixel resolutions – although most camera makers are now acknowledging the need for better low light performance. Low light and super high resolution imagers are generally at odds with one another.

Canon introduces new “low light” camcorders

Canon today introduced a new high end consumer camcorder line that features 1/3 sq in CMOS sensors – but having only 1920×1080 resolution. Rather than the typical gigapixel sensors being put into many consumer camcorders or still cameras, they have opted to go with a larger sensor having fewer pixels. This means each pixel’s surface area is larger and can capture more light. That in turn means the new cameras will work much better in low light and have greater dynamic range.

Looks like the new cameras also shoot 24p, in addition to 30p (wrapped inside a 60i stream) and 60i.

More information here – watch the Canon video explanation.

CMOS versus CCD video imaging and the “Rolling Shutter” problem

Older video cameras used CCD-based image sensors. For various reasons, that I will explain in a moment, camera makers have largely switched to using CMOS-based image sensors on low end and even some low end semi-pro video cameras (like the Panasonic HMC-40). All of the new digital SLRs (DSLR) still cameras that also shoot video using high resolution CMOS sensors too.

But there’s a nasty problem with CMOS-based sensors that can cause the video image to resemble a shaky bowl of jello, as illustrated in this video comparison between the Canon HV20 (CMOS-based) and the Panasonic SD5 (CCD-based):

The problem is that the CMOS image sensor is read “line by line” from top to bottom.  If the image changes during the read out, then one line may be slightly offset or shifted from the previous line.  This is known as a “rolling shutter” and creates the peculiar “jello effect” since not all lines (or rows) in the image are lined up with all the others.

Old movie film cameras used a physical shutter that open and shut exposing an entire frame all at once. (Although, there are some issues with that too which we can ignore for now.)

CMOS reads the image row by row while CCDs read the entire image all at once and do not use a “rolling shutter” – hence, no jello on CCD cameras.

So why use CMOS instead of CCD? The basic reason is that CMOS uses less power and produces less heat and is less expensive. As image resolutions have increased, the size (and cost and power) of the CCD imager has gone up.

Camera makers, especially in the consumer market, are in an arms race to each have cameras featuring ever more pixels. This means most have switched to CMOS because, apparently, most consumers do not care about the jello or do not encounter it often enough.

In the DSLR world, most any camera with 10 or more megapixel resolution is CMOS. CMOS works great for most still photography. But when these DSLRs are used for video, they too suffer from awful “jello”.

There are other things to consider too – CCDs may show vertical streaks in photos when there are very bright lights (sun reflected on water, stage lights, etc) in the image.  For higher resolutions, CCDs need to be physically larger – and more expensive.

Enhanced by Zemanta