Hazardous Stereography

Ali Kazimi (2016)

The first challenge was simply the right tools. Stereoscopic 2D technological support and the resulting infrastructure have grown at an exponential rate. In mid-2010, even though I had access to a high-end rig and cameras through the 2D Camera Company a key partner in the 2D Film Innovation Consortium (2D FLIC), they had a limited supply of high-end support systems—it is hard to believe but just two years ago, S2D monitors of any size were extremely rare even in a major international production centre like Toronto.

Trained crews conversant in, let alone trained in, stereoscopic 2D productions were virtually non-existent. There were just over a dozen experienced stereographers in the US and Canada, whose credits included the occasional live-action S2D feature films made since the eighties, theme park short films, IMAX films, corporate videos and a few music videos. It was not uncommon to get a bemused and perplexed response to the term “stereographer” from filmmakers, programmers, curators; some felt it was a pretentious term used to justify the passing fad of 2D. Indeed even among practitioners there was a great deal of ambiguity about the term. I posed the question on an international forum and got the most comprehensive response from Eric Deren, a visual effects specialist with a long resume in animation, who, like many of his contemporaries, had crossed over to live action: “The title of stereographer can be very wide… in most contexts it will simply be the person “who makes good 2D…”. Deren then went on to explain that the stereographer would ideally work on a film from pre-production all the way to post-production, conferring and advising the director, cinematographer, production designer, visual effects supervisor, the editor and finally assist the colourist with the stereo-grading of the completed film. Phil Streather, a UK based producer/stereographer added his definition: “Person with mathematical or intuitive knowledge of the 2D process.” In June 2010, I needed a stereographer, none were available in Toronto; my limited budget could not afford going Hollywood rates, even at a discount.

I have been interested in stereographic photography since experiencing the magical immersive yet miniature world in the Viewmaster in my childhood in India. As a documentary filmmaker and writer, I have collected and used stereoviews, as stereo-photographs were popularly called. Looking back I can see that I drew upon these visual stereoscopic experiences when I used digital technology to create and explore depth in my feature documentary Continuous Journey. Continuous Journey (2004) is a feature length personal investigation into the turning away, in 1914, of the Komagata Maru—a ship carrying 376 would-be immigrants from India. There are less than 25 known images of the actual events and participants. I used digital technology to integrate ephemera (stamps, military medals, passports), newspaper clippings documents, photographs and film in digitally created virtual 3-D space.

I wanted to create both depth and visual pleasure. Sometimes photographic elements from multiple sources were often reconstituted. This not only allowed for new compositions to be created but also for the incorporation of cinematic techniques such as “pulling focus”—which normally occurs in live action cinema. Working with my editor, Graeme Beck and visual effects creators, we placed cutouts of historical figures in the foreground, mid-ground and background. Then I directed a virtual camera with all the real parameters of lens, aperture and the resultant depth of field to create, for instance, crane shots complete with a pull focus from back ground to characters in the foreground.

In another case, the camera tracks through an image of dozens of new arrivals. From a single photograph, individuals and/or groups were cut out and placed in distinct planes in 3-D space. The virtual camera then tracked and shifted focus between each plane. A photograph of people huddled around a fire, is not only split into many planes in three dimensions but a 3-D moving image of a fire, complete with smoke, has been added in colour. In many ways these experiments of creating 3-D images in a 2-D medium have laid the groundwork for my exploration of similar and different techniques in 3-D cinema.

The excitement lay in creating depth on the desktop using Photoshop and After Effects. The digital revolution had started to transform independent low-budget filmmaking and this was the beginning. Five years later in 2008, I started researching stereoscopic 2D filmmaking that was once again being propelled by the digital revolution, the learning curved seemed steep and there were limited learning and practicing tools at hand.

The current stereoscopic 2D is propelled by the exponential rise in digital technology in film production, coupled with the phenomenal success of James Cameron’s Avatar. Much if not all the of the growth in S2D can be credited to Cameron’s championing and use of digital S2D. However, what is often forgotten is that Cameron himself did not come to S2D overnight, he spent the decade before Avatar experimenting with making underwater documentaries with different degrees of success. The stereography in Avatar though conservative was comfortable for its near 3-hour length, and this comfort no doubt was a considerable factor in its overwhelming success. Avatar not only resulted in rush towards more S2D films but also better rigs, and the worldwide conversion of tens of thousands of theatres to digital stereoscopic projection capability.

This has happened due to a perfect convergence of economics and technology. The studios wanted a greater draw for cinemas and digitally equipped cinemas can be converted at minimal cost to project 3-D. However it has remained my contention that because the vast majority of a Hollywood films revenues do not accrue from theatrical sales, but ancillary sales (DVD,TV, iTunes, airlines etc.—overwhelmingly 2D), these cinema interests demand S2D films to screen able in the 2-D. I am convinced that this multi-format packaging prevents S2D Cinema to develop its own true language; for it requires, I conjecture, a different, perhaps slower pace, rhythm and cadence. With the exception of IMAX large format films and Wim Wenders “Pina” There has been little work done in stand-alone S2D projects. The current films tend to be essential “flaties” with depth thrown in for good measure.

Stereography

As I mentioned earlier, the stereographer is the new person in the mix. Let me step back and give you an overview of how the human visual system works. It starts with a Stereopsis. Stereopsis is the complex process through which the brain combines or fuses the different 2D images from the left and right eyes to create the perception of depth. Technically, S2D camera systems mimic human physiology and try to replicate the way we see. A typical S2D rig will use two cameras each offset by a certain distance, called the inter-axial (IA) distance, to generate two identical images from slightly different perspectives, similar to those between our two eyes. The images have to be in perfect sync with identical focus, depth of field, colour and contrast, this is easier said than done. The mechanism for shooting stereoscopic 2D, known simply as rigs, therefore consists of two cameras either side-by-side or at right angles to one another with a partially silvered mirror at 45 degrees in the middle. The digital cameras then send two data streams that are separately recorded.

When one sees a S2D image on a screen without glasses, the left and right images appear to be overlaid with an offset or disparity between them. This to an experienced eye much can be gleaned from looking at such an image starting with the depth within the shot. The distance between the two cameras, the IA determines the depth. The point at which the disparity between the two overlaid images disappears is the simply the point of convergence. Convergence is where the screen plane lies.

The depth of a shot, its depth budget is often expressed as the percentage difference of the offset between the left and right image, and the total width of the screen. In the case of a 46-inch screen—the width is just over 100 centimeters, so a shot that has a maximum on-screen offset of 1 cm is said to have a 1% depth budget.

One of the things that cannot be fixed in post is the depth in any given shot—this is determined by the Inter-Axial (IA) distance between the two cameras, in simple terms the greater the IA the more the depth (note: if one pushes this too far you get miniaturization, an effect specific to S2D or conversely gigantism). What one can adjust in post-production is the point at which both cameras converge. If your depth, as determined by the IA has not been calculated properly, the scene may be unwatchable and the only way to fix it would be either to discard one of the camera images and keep it in 2D or to go for a 2D to 2D conversion.

The ubiquitous, often maligned, 2D glasses ensure that only the left eye sees only the image from the left camera and the right eyes receives the image shot by the right camera. From that point on, the brain takes over-using stereopsis to fuse the two images to create depth.

However, stereopsis is only one way in which in the human brain perceives depth. We also use a number of other visual cues, called monocular cues, such as perspective or the familiar size of objects to determine spatial relationships. A detailed look at some of the most successful early twentieth century stereoviews, reveals that what appears to be an overly cluttered 2D image is dramatically transformed by stereoscopic viewing into an effective S2D scene with points of interest distributed along the z-axis. Similarly, atmospheric effects such as smoke, rain or snow work wonders in revealing spaces.

Depth in space and determining which side of the screen certain elements will fall is only the starting point of stereography. Another vital consideration is what I refer to as the creation of depth within an object, without which it would appear to be akin to a cardboard cutout. However with a careful determination of the right IA with a given focal length of lens a stereographer can not only create volume and roundness but also control these for a desired effect.

During the entire process stereoscopic depth for every shot has to be precisely pre-visualised, calculated and executed since it cannot be changed in any way in post-production. Secondly, the depth of every shot has to be created keeping in mind the visual style of the film, this is known as the depth map or depth script of the film. While depth scripts are the norm in animation, in live action some have used storyboards with different colours to indicate whether the objects lie in front of the screen or behind it. For example, a blue outline might indicate that the figure is placed in positive space or behind the screen, and a red outline would indicate the object/person or part of it that would appear in negative space, or in front of the screen. The calculations for depths in both shots and the overall film require applied knowledge from psycho-physics of perception combined with an intuitive feel and an understanding of the narrative intent of the shot.

While our eyes can converge up to a point, hence allowing us to perceive objects appearing in front of a S2D screen, they are severely limited in their ability to diverge so in poorly calculated shot, the left and right eye images are so far apart that they cannot be fused into a single image. This will result in discomfort, manifesting as eyestrain to varying degrees. Similar visual stress may result when cutting; if from one cut to another the depth changes radically, this will also result in the audience feeling fatigued.

Hence it is essential for a stereographer to have as good a grasp of the psycho-physics of perception as possible. This is key, it is the responsibility of the stereographer to make decisions on every shot to best achieve the goal of the depth script while minimizing or completely removing the need for intensive, very expensive and at times impossible fixes in post-production. Finally, the stereographer will supervise the stereo-grading of the finished film, this is the final polish for a finished film where depth changes between shots are carefully smoothed out.

Stereography is integral to the way a film is photographed and edited, in cinematic terms how various shots are edited together is basic film language. Exploration of this theme is of special interest to me, working closely with vision scientists Rob Allison and Laurie Wilcox. I believed, there is need for an analogous but necessarily distinct cinematic vocabulary for describing an S2D scene, that parallels to the film language of long shot, medium shot, close up etc., but which takes into account depth together with the volume and roundness of people within it. Other issues involving depth concern its relation to convergence, i.e. the point where the screen plane lies in a shot and viewing distance. Essentially, the current terminology is largely technical and often uses borrowed and incorrectly applied scientific terms (and even there, inadequate). In short there is an absence of a spatial vernacular for filmmakers wanting to describe a S2D scene

In terms of both composition and pacing there is much that is still unknown, filmmakers have to learn how to see the world around us with the z-axis in mind. At the Digital Cinema Summit, in April 2009, S2D held centrestage. During his presentation, in a dramatic, hyberbolic gesture, veteran stereographer and author, Bernard Mendiburu declared that while 2D cinema has been around for several years: “How much do we know about the language of this medium?” . He asked rhetorically, after a brief pause, he changed the slide in his presentation—it had one word on it—“Nothing”. Mendiburu while being provocative, was cleverly drawing attention to the many unknowns of the potential. I feel he was calling for a degree of rigor in exploring the full potential of the medium rather than seeing it purely as the “golden goose of Hollywood.”

Beyond Technique

In 2007 during the Q & A at the Toronto International Film Festival, the first question for Agnes Varda after her astonishing documentary The Gleaners and I—was about the kind of compact digital camera she used. Varda shot back “Technique is boring, lets talk about content”.

Stereoscopic cinema has been intrinsically tied to the idea of the spectacular. The “classic 2D” moments are generally described as those in which something popped out of the screen or appeared to come towards the audience. Not surprisingly the vast majority of S2D films are horror, fantasy or science fiction films; in 2011 it appeared that there might be a change in the offing with films such as The Great Gatsby and The Life of Pi.

I wanted to get beyond the spectacular use S2D but at the same time make a stand-alone S2D piece and test out my hypothesis that such a film would not entirely work when viewed in 2D.

My intention was to use the new digital S2D to continue my on-going exploration of race and representation socially. My explorations have led me to reflect on the iconic Canadian landscape, it is with in this space that notions of settler (read: white) and immigrant (read: racialized), Canadian and New Canadian emerge. Add to a sprinkling of post-9/11 paranoia and Islamophobia you have a potent mix. Even before 9/11 brown male bodies particularly with identifiers of Islam were often viewed with suspicion in Canada and the United States, at times as threat to national security. I had written a dramatic feature film script in part based on personal experiences,and others gleaned from interviews and media stories. The financing for Agar never came through, however I did have small arts council grants, I rewrote a short story based on this, and designed it for a S2D shoot.

The narrative was simple—nineteen-year old Zakir Ahmed, sets out to run an errand and start shooting a film, that he making as gift to his father for Father’s Day. In the security obsessed post 9/11 atmosphere, for a young man of South Asian descent, even the most ordinary actions become extraordinary. Zak, as he is also known, is seen filming a power station, one where his father used to work. Dressed casually in a grey t-shirt and jeans, he wears a black and white, kaffiyah, a middle-eastern scarf. On the way to his shot he runs an errand for his father, dropping off a box of electronic household waste stored in an old cardboard box. Radio reports of Canadian soldiers killed in Afghanistan are followed by reports of a police foiling a terrorist attack by “East Indian Muslim” men who were allegedly caught trying to build a dirty bomb.

Seeing the hand-written Arabic script on the box dropped off by Zak, the waste station employee runs a Geiger counter over the taped box and gets a signal indicating radioactive material.

As he films through a chain link fence on his 16mm Bolex camera, he is arrested by two police officers. He is interrogated, asked to strip naked and wear an orange jump suite. The narrative is driven by a first person voice over. Zak had run over the box as he backed out his car from the garage, unbeknownst to him he had crushed the old smoke detectors that were in the box. The radioactive reading had been generated by tiny amount of Americium used in the detectors.

The film ends with Zak in his downtown Toronto home looking straight at the audience.

(Ironically a few weeks later, the G20 protests happened in Toronto. Anyone attempting to photograph the security perimeter fence had their cameras seized. Arbitrary detention became the norm during those fateful days. Men from racialized groups were specifically targeted, some were ordered to strip and wear orange jumpsuits.)

…and back to technique

I had anticipated that the S2D would be challenging but little did I know it would be such a technically demanding film in many ways. As mentioned earlier, I had chosen to be my own stereographer, I had gleaned much from workshops, panels, books and my discussion with my colleagues, visions scientists Laurie Wilcox and Rob Allison. I had armed myself with a couple of S2D apps including the RealD Stereo-calculator (at $300 probably the most expensive download at the time). I had made a story-board with depths indicated in different colours, and ran these by Allison and Wilcox who assured me that all the ideas were doable. I had equipped myself with stereography apps on my smartphone such as the one by RealD. These have computational formulas and give read out, based on the geometry of a scene, i.e. distance from camera to nearest object and the farthest object. The readings determine the inter-axial distance to be set for a given scene to create the desired depth.

However, as with any technical tool, this is only the starting point. While stereography is the lynchpin of stereoscopic 2D cinematic practice and the skills of the stereographer lie at the intersection of art and science, the demands of the story has to drive the process.

Psycho-physical studies using eye tracking devices have shown that when audiences are shown images of individuals in both 2D and then in 2D, they view the visual image differently. In the 2D version, viewers spend most of their time looking at the face and body of the photographed person. When shown the 2D version of the same image viewers spend more time exploring the space around the person.

One can draw at least two inferences for S2D cinema, and both of these continue to be continually debated. First, since viewers are being given more experiential information and their stereoscopic visual experience is being enhanced by the already spacial-ized sound (surround or at the least stereo sound) they would want to spend more time on each shot. Hence the pace of the film would be slower. Secondly, the contentious issue of deep focus. If viewers are exploring space in any given shot more than what they would in 2D, one would assume that it would be best to shoot with deep focus. (Many have argued that Citizen Kane with its iconic deep focus photography by Greg Tolland is a perfect touchstone for S2D cinematography. Attention is drawn to characters using very dramatic, chiaroscuro style, high contrast lighting). However, if viewers are shown shallow focus stereoscopic images with an actor dramatically in focus while everything else around her is soft, it is likely that viewers would get fatigued as their visual system tries to resolve out of focus details. Often the rebuttal to this that if the film is paced properly the audience will not have the time to look around, which of course takes one back to the first inference—and the debate rages on.

For Hazardous, I chose to use deep focus and a slower pace.

Motion, either within the shot or as a moving shot—whether it is a pan or a dolly, accentuates depth perception. I decided to incorporate as many moving shots as time would permit. Since the narrative was very much about Zak’s body within the urban post-industrial space of Toronto, in addition to depth, roundness and volume were important. I wanted most of the drama to be behind the screen. I devised a subtle conceit to the stereo design. When Zak was in his “creative zone” his body would push out of the screen towards the audience, the intent was that this would draw the audience closer to him. The depth budget of most shots was fairly conservative, between 1% to 2.5%. The initial design of the film included Zak’s first person address to the audience, while sitting in his living room, here in the safety of his home Zak leans out of the screen into the audience.

A key parameter in determining the depth of S2D films is the maximum size of the screen on which the screen would be shown. Geometry dictates that a stereoscopic film designed for a large screen can be screened on a television monitor (albeit with diminished effect), the converse is not possible. It was my hope that Hazardous would be screened in festivals, so I chose a 40-feet screen as the final screen width.

I had chosen the locations so that most of the action would be outdoors, in relatively quiet locations. A west-end residential street and back alley in downtown Toronto, a recycling depot at the garbage incinerator and a derelict dead-end street with an abandoned power- generation station along the eastern fringes of city harbour. Each shot would allow for some experimentation with depth, composition and motion.

Production

Given the dearth of technical equipment from production to post-production, there had been no opportunity to shoot tests, and nor was there any room for error in getting the material since the equipment was available only for a brief window. The crew was made up of a mix of undergraduate production students and recent graduates; Rozette Ghadery, Iran’s first female director of photography, at the time a MFA candidate at York, was the cinematographer.

The first day, a fifteen-member crew watched and waited as camera assistant and 2D tech did his best to try and align the camera rig—for nearly seven hours. The multi-talented, Owais Lightwala, then a second year theatre student at York, who played the lead, waited patiently as I deliberated whether or not to pull the plug. In spite of tech support from the supplier and the generous emergency input of Toronto based stereographer Brent Robinson, the cameras were impossible to align precisely. A vertical disparity remained. We only had a very basic of S2D monitor (a 26-inch computer screen) and were limited to a single pair of unmatched, albeit high-end cinema lenses. In addition, the cooling system for the bulky data digital recorder was inadequate for the hot summer temperatures. The Data Wrangler, as it was called, had to be periodically cooled by radical measures; taking off its cover and spraying compressed air at close range.To add insult to injury, every time the rig was moved for a new setup it would be thrown out of alignment and it would take over an hour to get it ready for the shot. I shared this experience with Wim Wenders over lunch, “I believe it”, he said simply. “I have been there.” He agreed that just a year later the technology and processes had made it possible to set up faster.

The technical limitations changed how much could be shot; entire scenes were dropped and coverage was compressed by designing “oners”—complex moving shots that included in one case a boom down from about 8 feet to three, combined with a 270 degree pan followed by a lateral dolly move and pan. The stereography was nerve- wracking, I was anxious about the shots with buildings at near infinity, fearing that the background would be divergent and viewers would not be able to fuse it.

This was the first time I had shot an entire film with a single focal length lens. In S2D wider lenses are preferable since they retain roundness and volume. In 2D cinema, longer, telephoto lenses are often sought for their ability to compress foreground, mid-ground and background, however in doing so they also necessarily compress the depth within an object. A telephoto lens would be tricky to use in S2D, as it tends to create the normally undesired “cardboard cut-out” effect. The 12mm lens on our cameras had a field of view equivalent to a wide-anglish lens if we were shoot 35mm motion picture film.

Viewing the rushes was a thrill and relief, all shots worked in S2D and every thing was comfortably fused. I could have pushed it further, but it a remarkable start. I had taken to heart some of the suggestions made by Laurie Wilcox, a side shot with hydro-wires receding into the distance cuts through and reveals all the planes of depth, we deliberately dirtied the glass of the interrogation room so that it reveals itself as a barrier when the camera sees Zak alone inside the room.

The off-line edit was done using a normal monitor with anaglyph (red-blue) glasses. In the edit the limitations of the script started to reveal themselves. The direct address to the viewers did not seem to work at all. As a S2D shot it was one of my favorites, it had the largest depth budget, the character was palpably leaning into negative/audience space, it felt like we were there with him in the room, yet cutting back to him made the other scenes feel illustrative and redundant. Joe Crummins, the editor and I cut the film while watching it in 2D, the same cut in 2D felt off in places, as if the edit points were in the wrong place, as if the entire pace was out of step.

Getting it out

Wenders was prophetically blunt about the prospects for the film, “ you are dead in the water with a 2D dramatic short.” Again he spoke from experience, his S2D short drama made in 2009, was screened in a handful of festivals. I received several compliments from seasoned stereographers regarding the film clips that I showed during my presentation at the first Toronto Stereoscopic 2D Conference in June 2011. A comment during the Q&A drew attention to the fact that the film was also an anomaly in terms of content, a political film dealing with race.

I called a major film festival to find out what kind of S2D format they would like me to submit the film in, since the application form had no details. I was put on hold, they came back and said “send us the film in 2D and if it is selected we will screen the film in 2D”.

It was the Hollywood approach—depth is simply an add-on. I did wonder if they could also say send us the film without sound and if selected we will play it with sound” or “in black and white—if selected colour”. Some programmers did call me back, and offered detailed and seemingly expert opinions on why the film was not selected: “Did not see depth, not that they were looking for 2D moments of course but it seemed not….”

It is clear to me that we are still in a period, where despite protestations to the contrary, spectacle is still an essential expectation particularly in S2D shorts. It reminds me of the early days of stereophonic sound, we would gather in room to listen to the magic hearing the spatial magic in the stereo recording of Pink Flyod’s Dark Side of the Moon. Now we take stereo for granted it been superseded by surround sound not only in the theatres but also in the home. The vast majority of audience members forget that immersive and all-enveloping nature of cinema is sound; remember film is an audio-visual medium, sound is the other half of creating space. We have taken for granted that we will hear voices and sounds located in their proper positions in a given space; that as car moves from left to right the sound will follow it. The picture is finally catching up to the sound, we are still trying to get used to it, the spectacle will take a while to fade away so that we can simply be immersed in a story not just aurally but visually in all its attendant depth.