First Ever Surgical Video in 4k Ultra High Def- Hollywood and Doctor Team Up for Revolutionary Projects on Future Visualization in Medicine

our digital cinema control room- Sony controls

For the first time ever Hollywood’s digital vanguard has teamed up with a surgeon to use tomorrow’s cinematic tools to revolutionize visualization in the operating room. We successfully filmed produced and played a 4k digital cinema movie of a live human surgery.  This was an amazing collaboration between a fertility surgeon, the AAGL endoscopy society, Sony Medical & Broadcast divisions, Red Digital Cinema Camera Company, FotoKem, Vincent Pace films, USC professor Richard Weinberg, and the NBA just to name a few!  We also showed the largest medical 3d displays ever.

The AAGL reported on this groundbreaking session today .   Further details are here and here.

Dual Sony SXRD 4k projectors in 3D scaffold

New cinematic technologies are transforming the film business today.  The two major revolutionary developments are 1) ultrahigh definition “4k” technology - which brings resolution to 4 times that of HD and 2) realistic immersive high definition 3D. I set out to introduce these technologies to the medical world and to see if we could for the first time apply them to surgical practice- setting the goal to once again use technological innovation to improve our patient outcomes.

Virtually all of my surgery is endoscopic- performed through tiny telescopes and viewed on a TV monitor. In 2001 I performed the world’s firs HDTV surgery and demonstrated how increased resolution improved the surgeon’s visualization and performance of procedures.  For those interested in the history of HDTV surgery and the details of its development further details of my work are here from the New York Times and here from Science Daily and here from MIT Technology Review.   Over the past two years I refined this work with even better performing camera systems and this work was featured on 20/20 and on the National Geographic Channel’s first ever HD medical show - Inside the Living Body.

Hollywood is embracing its digital future by adopting (with $1 billion in financing and a follow-up deal by Sony) planned conversion of 20,000 theaters to ultrahigh definition 4k (4096x 2048) video.  The revolutionary Red One camera is one of the few that can natively record this type of file.   Having heard about it I went to vegas for NAB in April and saw with my own eyes the amazing realistic movies  being made with it.  While there I met with Ted Schilowitz, Red’s “Leader of the Rebellion” and was ecstatic with his desire to help with with a proof of concept- to film surgery with a 4k camera. I also partnered with Sony’s Medical and Broadcast Divisions who eagerly helped me set up a 4k and 3dHD digital cinema in an international medical conference hall in the Las Vegas Paris Hotel.  The theme of my Keynote lecture was “Film and Medicine: From the Past to the Future- Through the 4th Wall.”  In this, I traced how cinematic film  technology has transformed the teaching and performance of surgery- and how the future will again have revolutionary changes in physician visualization.  Our Hollywood partners allowed me to first show jaw-dropping pro-sports and theatrical demos of the technology.

Sony’s digital cinemas allow the next generation of film to be projected at these previously impossible resolutions.  We used dual SXRD 4k projectors with linear polarizing filters for 3D HD then went to a single projector for the 4k.  4k footage has to be played off a dedicated media block digital server  and we used an LMT-100 like a movie theater would and FotoKem converted the 4k files into a DCP digital cinema package that can be played.  For 3d footage we played off an HD SR tape dual stream deck.  I’ll post a follow-up explaination of each of these technologies in detail.

By increasing resolution to this level we allow the surgeon to be actually immersed in images that surpass the live surgical experience. The progress from regular surgical film technology is like comparing sitting in an HD home theater to watching a video on a cell phone.

Ultra high resolution digital cameras are transforming the art of cinema. Leading Hollywood directors such as Peter Jackson and Stephen Soderbergh today have just started filming the next generation of cinema blockbusters using cameras with “4K” resolution, four times the resolution of High Definition (HD) with 4096 lines of resolution to give audiences unprecedented realism. The Red One digital cinema camera is the at the forefront of the revolution.  Director Soderberg previously described this technology as “This is the camera I’ve been waiting for my whole career: Red is going to change everything….Shooting with Red is like hearing The Beatles for the first time. Red sees the way I see.” 

Amazingly, the surgeons in the conference were able to visualize the surgery they were watching better than if they had been in the operating room live. If it can transform the immersive experience of the movies with unprecedented realism wouldn’t you want that degree of vision in your surgeon’s hands? By combining unprecedented resolution and magnification the surgical images were beyond what a surgeon would have standing live in the operating room. Those in the audience predicted this technology would further revolutionize minimally invasive surgery as it becomes incorporated into the OR of the future.

We then projected the largest high definition (HD) 3d surgical images ever. Wearing glasses reminiscent of today’s 3d Hollywood blockbusters U23D and Journey to the Center of the Earth a standing-room only crowd of 1600 surgeons felt as if they could “reach out and join the operation”. These images were enabled by converting Sony’s ultrahigh definition and 3d theatrical systems to show medical footage in what the AAGL called the “Theater of Tomorrow.”  A Vince Pace 3D demo reel showed where Hollywood is going as well with Gwen Stefani concert footage, underwater vistas, and movie clips.

Since laparoscopic surgery is performed by a surgeon watching on a video monitor making the images “better than live” may facilitate teaching and procedures in the future.

The 4K system, manufactured by RED Digital Cinema Camera Company, was used to film Jumper, Crossing the Line, and The Argentine. This recording represents its first use for medicine and biology in the world.  In addition to a surgery we showed the first ever 4k microscopy images of single cell organisms ever recorded (from USC).

Stay tuned a for a series of posts coming up this week that will review the session, how we managed the collaborations, the difficulty in executing each of the technologies, and where I predict this is going in surgical visualization.