Lightfield Photography Part 2 (Lytro Illum)

Lytro Illum  (Photo credit: KoFahu meets the Mitropa)

I talked a lot in the previous part of this article about how light field photography works and why it's useful. In this part I'm going to focus primarily on the enhancements to the Illum which is the recently announced second generations Lytro camera. I'm going to focus mostly on why the enhancements in the Illum are potentially useful even for serious photographers.

As I opined before, too much emphasis is put on the Lytro's ability to refocus. The fact of the matter is that this ability to refocus comes at a cost. I've often referred to the first generation Lytro as a great proof of concept and it was; but it wasn't without some fairly significant faults. Those faults caused a fair amount of consternation in some circles and are likely causing Lytro some headwinds right now. If you read any of the comment sections associated with articles written about the Illum you'll know what I'm talking about.

In my opinion Lytro made a mistake in their marketing of the first generation camera. That mistake was in focusing too much on the megapixels captured while avoiding at all costs any mention of the resolution of exported 2D images. On the one hand this was understandable. 1.1 megapixels stopped being an impressive resolution a decade or more ago. The limited resolution was also at odds with their desire to present the Lytro camera as a premium product at a premium price. In addition it was generally difficult to get a good quality picture even at the 1.1 megapixel export resolution. Pictures were often soft and colors were subdued and muted in my experience. It was rare that I ended up with a final image that wasn't mediocre at best. This is not a good outcome for a camera that has as its major selling point the ability to refocus so that you can avoid bad pictures.

In addition there were aspects of the hardware that bugged me. The biggest was the LCD. The small dimensions were inevitable given the industrial design but the quality was less than stellar. It was dim and had poor viewing angles which made the camera difficult to use. My other hardware related gripe is the lens cap. It never stayed on as the magnet was weak. Consequently I was always losing it. Having said that, I loved the concept and the technology and have been looking forward to Lytro's second generation camera for awhile now.

Enter the Illum, a product that on paper promises to be a significant improvement over its older sibling. The design is slick and modern looking with a definite nod to digital SLR's.

The lens is a permanent part of the body and features an impressive 30-250 mm zoom range with a constant F/2.0. What does that mean in layman's terms?  Basically you get everything from a fairly wide angle to an 8x zoom with excellent light capture across the entire range. The ability to capture light is important as it relates directly to the quality of the picture. This is true of any camera but even more so in the case of light field photography.

Our eyes and brain are a great team. They work in tandem to create a visual experience that is generally smooth and glossy. Normal camera's don't have a sophisticated human brain doing real time processing which is why it can be difficult to get a good picture.

Getting a well focused image requires an appropriate depth of field, (AKA focus point) and either a whole lot of light so that the exposure time is short or a tripod and long exposure time. The Illum doesn't have the human brain but it does have a whole lot of smarts and technology built in that should make it easier to take good pictures.

The first advantage the Illum has over most cameras is that 2.0 F-stop rating. The thing to know about F-stops is that the lower the number the faster the lens. Put another way, as the number decreases the lens is capable of letting in more light because it can open wider in a given period of time. Normally that is a bit of a trade off as more light means a shallower focus range. Large F-Stop numbers mean that most or all of a picture you take will be in focus while small F-stops mean you'll have a steadily narrower focus range as the F-stop number decreases. This isn't a problem for the Lytro as capturing the light field allows an image to be rendered that is in focus over a much wider range of distances. In effect you get the wide depth of field advantage of a high F-Stop but with the high light capturing capability of a much smaller F-Stop which should translate into better quality pictures in situations such as sporting events where fast shutter speeds are needed.

The Illum's 40 mega ray censor will capture roughly four times the light rays that the original did. This in turn translates to 4 mega pixel 2D images. That 4 mega pixel export number is likely a compromise that provides reasonable output in a wide range of scenarios. Macro images and images a very limited depth of field could likely be rendered at higher resolutions with good results as the light rays captured would originate from a much narrower depth of field range and thus provide more data(light rays) at a given focus depth with which to render an image. Of course the inverse is true as well. The wider the available focus range the less data will be available at a given focus point to render from. This may explain why Lytro images can look soft. Lytro basically has two choices when they don't have a lot of data at a given focus point. They can interpolate between available light rays or look at light rays that may provide additional information but not be ideal for a given focus point. In all likely hood I suspect they do a bit of both.

The LCD may be the biggest upgrade of all though. As I mentioned above, the original Lytro's LCD was... sub optimal. The dimness, limited viewing angles and small size meant that I was often shooting blind. The Illum's much larger screen and ability to tilt will inevitably lead to a better experience. How much better remains to be seen but I'm fairly optimistic. My Cannon EOS 70D has an LCD that rotates and moves in a lot of ways that the Illum's won't but it won't tilt back without rotating it 180 degrees out from the body. This provides a similar but not identical experience and after trying it out I think I'm going to like the Illum a lot. The best shot is often taken from lower than eye level, particularly when you're tall as I am. Being able to look down and easily see the framing for the shot is very useful.

The UI on the original Lytro was spartan. That was because of the tiny touch screen and  limited processing power. The Illum's Snapdragon 800 processor has four cores which will enable a much better user experience. One feature shown in the pre production models is the ability to visualize what will be within the refocusing range. That alone is worth the price of admission. It was always frustrating to me when I'd take a picture, load it into the Lytro management software and then not be able to refocus as I wanted.

Of course the big question is will all of this be worth four times what the original Lytro cost? I pre ordered mine and as an owner of the first generation camera I got an additional ~$250 discount. That was good enough for me to pull the trigger so I'll be able to explore first hand what this camera does in another month or two. I have some thoughts on the pricing that I'm going to reserve for the next part of this series.

Related Articles:

• Lytro's Illum Can Do Something No Other Advanced Camera Can
• Lytro's new Illum camera brings light-field tech to pros (and it could change photos forever)

A Brief Overview of Lightfield Photography Part One

The following is an update of an article I wrote for Make Magazine. A much slimmed down version with some great graphics appears on pages 60 & 61 of volume 38. The online version which resembles what is below much more closely can be found here.

The Lytro camera has been around for a couple of years and the feature most people seem to talk about is the ability to fix or change the focal point of a picture after it has been taken. But that isn't the whole story; in fact it’s just the tip of the iceberg.

Traditional cameras, whether digital or analog capture a scene from a single point of view. Photoshop and similar software enable amazing things to be done to these images but it’s unlikely that any amount of post processing will give photographers the ability to slightly change the perspective of an image, change its focal point or render it in 3D. To do those things you need to have captured the entire “light field”. The light field is made up of additional data types that include not only the color and intensity of a light ray but also the direction it came from. In addition the light field is all of the light rays that enter the camera, not just those that are focused on a traditional camera’s sensor or film to create a single point of view 2D image.

The first generation Lytro captures the light field by directing the light rays that enter the main lens onto an array of over one hundred thousand micro lenses. The micro lenses in turn direct the light rays onto a  6.5 x 4.5 mm CMOS sensor with the ability to capture eleven million light rays arranged in a 3280 by 3280 grid. Each micro lens utilizes a roughly ten by ten pixel portion of the CMOS sensor.

Capturing the light field is only the first step. The next step is to generate an image that can be viewed. Doing this has been described as “ray tracing in reverse”. To explain what this means I’m going to describe how pinhole cameras work, traverse briefly through ray tracing and finally explain the Lytro rendering process. All three have two things in common. First there is an observer viewing the scene and second, the scene must somehow be rendered onto a screen that the observer can view...

In a pinhole camera light rays pass through a hole in the front of the camera and appear on the opposite surface as an image that is upside down and reversed. The back wall is essentially a screen. If you were inside the camera with your back to the pinhole lens you would see the projected image in front of you.

In the case of my crude illustration above try to imagine if you can; that the scene object is a cactus. This and all other images can be clicked on to get a higher resolution version. Sadly the quality of the drawings does not improve.

In ray tracing a scene is described mathematically using geometric shapes, textures and light sources. A point outside the scene is selected that represents the position of the observer and an image is generated from the perspective of that observer...

Ray tracing differs from the pinhole camera example in three ways. First, the scene does not exist in the real world and has to be rendered. Second the “real” scene is in front of the observer rather than behind and finally since the scene is virtual, the image on the screen needs to be rendered. This is done on a pixel by pixel basis. “View” rays are cast out into the scene for each pixel and the color of the pixel is calculated based on the objects and light sources each view ray hits while traversing the scene. Sending view rays from the observer greatly reduces rendering time as only the portions of the scene visible to the observer need to be calculated.

In the case of the Lytro we have the stored light rays that describe the scene captured when the picture was taken. In order to project an image onto our screen and generate an image a focal point needs to be chosen. Given the selected focal point, the Lytro software uses the stored light rays to render the image. This is ray tracing in reverse in that the rays projected onto the screen to generate an image have their origin point within the scene rather than having been projected from the observer and through the screen into the scene.

To review, in ray tracing the scene is rendered by shooting view rays out from the observer and through the screen while with the Lytro the scene is rendered by shooting light rays captured in the scene back onto the screen.

A recent addition to the Lytro Library software is the ability to create 3D images. This ability demonstrates another advantage of capturing the light field. As in the pinhole example imagine yourself inside the Lytro camera. Scooting around would give you a slightly different view of the external scene. The data needed to render those transitions is part of the captured light field and can be used to generate 3D renderings of the captured scene. A pair of inexpensive Anaglyph glasses is all you need. It’s easy to export the images as JPG’s so that they can be viewed by anyone with Anaglyph glasses. Below is an example based on a picture I took at the OK corral with my first generation Lytro.

One downside of the Lytro camera is the lack of a published API for image manipulation. Lytro uses a proprietary image format and while a lot of work has been done to reverse engineer it and create software to manipulate images as I write this there is no comprehensive cross platform API or software available for working with LFP’s (Light Field Picture files). The best resource I’ve found is “Lytro meltdown” (http://optics.miloush.net/lytro/Default.aspx) which tends to be Windows centric but contains a lot of useful information.

Light field photography is still in its infancy. Prices were initially high and the results underwhelming but things are starting to change on both fronts. The first generation Lytro camera can now be had for under $200.

The thing I’d most like to see is a comprehensive cross platform open source library that can be used to manipulate and manage LFP files. The Holy Grail would be software capable of taking a raw Lytro file and rendering images.

The recent announcement of Lytro's second generation Illum camera has some interesting technical and marketing implications that I'll talk about in the second part of this article.

Related Articles:

• The Clever Thinking Behind Lytro's Gorgeous New Camera Design
• In Brief: Lytro's Illum Looks More like Mirrorless than Monocular
• Lytro Goes After Prosumer Photographers With New Camera